This is a test version of Biostars. For the public version, visit https://www.biostars.org.
How to edit a fq.gz file

Hi everyone:

I would like to delate the last 15 letters from the 1st line of each read:

@A00261:889:HHW5HDSX7:3:1101:2528:1016 1:N:0:GCCAATATCT+AGATCTCGGT
GNTTGAATTCAATGTGAGCAGAAGCAAGCCAGATAAAACACAAACAGTAAATTAAGCTAAGTTCTGAAGAGCTTGGCTTCTGCCAATAGAGCAAACAACCTGGGCTATGTTAAATTCGCCTCTGGCGGCTTCAGTCTACATTACTAGAGG
+
F#FFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFF

@A00261:889:HHW5HDSX7:3:1101:3884:1016 1:N:0:GCCAATATCT+AGATCTCGGT
GNTTGAATTCAATGCAGAATGCTGAGAGCTTGCCACTTCTGGCAATTAACTTGAGATAAACAAAAGTGGTAAGAGGAGGCATTAAGTACCCACCTGCAGAAGACTGGCTCAGTGCTGAGTGCTCTCAACAGATGAGTGCTAATTGCAATG
+
F#FFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF,:F:FFFFFFF

Any help would be appreciated.

fastq
• 3,911 views • link
updated by Istvan Albert Istvan Albert • written by nicole.reguerar • nicole.reguerar

This sounds a lot like an XY problem (https://xyproblem.info/). Can you please explain what problem you're trying to solve? (more bases than quality string, removing library indices from read names, etc)

• 0 views • link
written by LChart LChart

are you planning on trimming the read name? you should provide an example

• 0 views • link
written by Istvan Albert Istvan Albert

Hi:

I want to edit the header line. I am looking for something like this:

original:

A00261:889:HHW5HDSX7:3:1101:2528:1016 1:N:0:GCCAA**TATCT+AGATCTCGGT**
GNTTGAATTCAATGTGAGCAGAAGCAAGCCAGATAAAACACAAACAGTAAATTAAGCTAAGTTCTGAAGAGCTTGGCTTCTGCCAATAGAGCAAACAACCTGGGCTATGTTAAATTCGCCTCTGGCGGCTTCAGTCTACATTACTAGAGG
+
F#FFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFF

result:

A00261:889:HHW5HDSX7:3:1101:2528:1016 1:N:0:**GCCAAT**
GNTTGAATTCAATGTGAGCAGAAGCAAGCCAGATAAAACACAAACAGTAAATTAAGCTAAGTTCTGAAGAGCTTGGCTTCTGCCAATAGAGCAAACAACCTGGGCTATGTTAAATTCGCCTCTGGCGGCTTCAGTCTACATTACTAGAGG
+
F#FFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFF
• 0 views • link
updated by Ram Ram • written by nicole.reguerar • nicole.reguerar

This requirement makes no sense. Why do you need to trim the header by position?

• 0 views • link
written by Ram Ram

To trim off 15 bases from every fourth line, starting with the first you can do something as simple as:

awk 'NR % 4 == 1 {sub(/.{15}$/, "")} {print}' test.fq
• 0 views • link
written by Istvan Albert Istvan Albert

3 answers

Adding this as an answer since it technically answers OP's question:

cat sample.fq
@SEQ_ID789012345678901234567890
GATTTGGGGTTCAAAGCAGTATCGATCAAATAGTAAATCCATTTGTTCAACTCACAGTTT
+
!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF>>>>>>CCCCCCC65

cat sample.fq | seqkit replace -p '.{15}$'
@SEQ_ID789012345
GATTTGGGGTTCAAAGCAGTATCGATCAAATAGTAAATCCATTTGTTCAACTCACAGTTT
+
!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF>>>>>>CCCCCCC65
• 0 views • link
written by Ram Ram

Following works as well (using OP's example to make it specific for their use case) :

$ sed '/^@A00261/s/.\{15\}$//' test.fq
@A00261:889:HHW5HDSX7:3:1101:2528:1016 1:N:0:GCCAAT
GNTTGAATTCAATGTGAGCAGAAGCAAGCCAGATAAAACACAAACAGTAAATTAAGCTAAGTTCTGAAGAGCTTGGCTTCTGCCAATAGAGCAAACAACCTGGGCTATGTTAAATTCGCCTCTGGCGGCTTCAGTCTACATTACTAGAGG
+
F#FFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFF
@A00261:889:HHW5HDSX7:3:1101:3884:1016 1:N:0:GCCAAT
GNTTGAATTCAATGCAGAATGCTGAGAGCTTGCCACTTCTGGCAATTAACTTGAGATAAACAAAAGTGGTAAGAGGAGGCATTAAGTACCCACCTGCAGAAGACTGGCTCAGTGCTGAGTGCTCTCAACAGATGAGTGCTAATTGCAATG
+
F#FFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF,:F:FFFFFFF
• 0 views • link
written by GenoMax GenoMax

True, seqkit works with gzipped files and writes gzipped out as well - not saying we can't do it with a simple zcat | sed '1~4s/..' | gzip > but seqkit wraps the nuts and bolts well.

seqkit replace -p '.{15}$' in.fastq.gz -o out.fastq.gz
• 0 views • link
written by Ram Ram

I would like to delate the last 15 letters from the 1st line of each read

If this means trim/shorten the sequences by 15 bases then you should be able to do this using bbduk.sh (look at forcetrimright=N option. Replace N with (your readlength -15 ). Other trimming programs should have similar options.

forcetrimright=0    (ftr) If positive, trim bases to the right of this position
                    (exclusive, 0-based)
• 0 views • link
written by GenoMax GenoMax

I think OP wants to partially (?) remove the I1/I2 barcoding content. They need the header edited, not the sequence.

• 0 views • link
written by Ram Ram

Editing fastq header like that does not makes sense but then OP may be the only person who can clarify their requirement.

• 0 views • link
written by GenoMax GenoMax

You're right. Like LChart said, this looks like an XY problem.

• 0 views • link
written by Ram Ram

Try this tool:

fqkit trim -r 15 your.fq.gz
• 0 views • link
updated by Ram Ram • written by size_t • size_t

Interesting tool. Did you do any bechmarking against existing tools such as seqtk and seqkit?

BTW, trim has a stub as far as documentation goes. It trims sequences and not headers, right? If so, the command is not relevant to OP's use case.

• 0 views • link
written by Ram Ram

i don't know why I can't add a reply, @Ram

It trims sequences and not headers, right? yes, this command just trim sequences and not headers, my bad.

• 0 views • link
written by size_t • size_t

seqkit has a tool to trim header right ?

• 0 views • link
written by akshay • akshay

Yeah, seqkit replace can be used with a '.{15}$' regex. I still think this is a really weird thing to do.

• 0 views • link
written by Ram Ram

Log in to answer this question.

More posts like this