This is a test version of Biostars. For the public version, visit https://www.biostars.org.
HOw to modify the description line 1 of a fastaq file to match with third line?

I am using paired end SRA data, it looks like this 

@HWI-ST915_0064:2:1101:1420:2104/1
GTCTCTTCGCACGCTTTCACTGTGAACGGTTCGGCATCGAGAAGGACGCAGTTCCTCTCCGGCTTGGACCAGTTTCTGGTGGCCACGGCTGCCCCCATCC
+SRR1188607.1 HWI-ST915_0064:2:1101:1420:2104 length=100
HDHHHHHHHHHHHHHHHHHHHHHHGHHHHHHHHHHHHHHFHFFHHHHHHHHEHEHHHHHHHGHHHED?EE=A@BACDDECCE@DB74?############

@HWI-ST915_0064:2:1101:1498:2108/1
AAAGATTGCAATGGAGGAGAAAGGGAAGACCCTGCCTGAAGAAATGCAATTGATAAATAAGTTGTTGTCTGAGGAAAAGGGTTCGGAGAGGATGAGAATG
+SRR1188607.2 HWI-ST915_0064:2:1101:1498:2108 length=100
GFHHHHHHHHHHHHHHHHHHHHHGHGHHHHHHHHHHGHHHHGHHHHHHDHGGHHFHHHHHGEGGFGGFGFHFHHDHHGHHGFGFGHGFHHHHHHHHHHHH

Othe read file is similar except with /2 at the end

For further analysis I need to match the description in line one with line three after +, but in line three there is extra information as SRR1188607. 1, SRR1188607.2 etc how to get rid of this so that it matches with line one description. Also how to delete everything after +

Thanks

fastq header modification
• 3,488 views • link
updated by cpad0112 cpad0112 • written by majeedaasim • majeedaasim

I wonder why you need to do this... most programs ignore the 3rd line...

• 0 views • link
written by Pierre Lindenbaum Pierre Lindenbaum

For example, the biopython parser throws an error if the third line is present and not equal to the first line. AFAIK per convention the third line has to be equal to the first, or absent.

• 0 views • link
written by WouterDeCoster WouterDeCoster

4 answers 

awk '(NR%4==3) {i=index($0," ");printf("+%s\n",substr($0,i+1));next;}{print}' input.fastq
• 0 views • link
written by Pierre Lindenbaum Pierre Lindenbaum

Pattern to exclude is

SRR[some_digits][a dot][another digit][a space]

sed combined with regex

sed 's/SRR[0-9]*.[0-9] //g'

Input

[Desktop]$ cat test.fq 
@HWI-ST915_0064:2:1101:1420:2104/1
GTCTCTTCGCACGCTTTCACTGTGAACGGTTCGGCATCGAGAAGGACGCAGTTCCTCTCCGGCTTGGACCAGTTTCTGGTGGCCACGGCTGCCCCCATCC
+SRR1188607.1 HWI-ST915_0064:2:1101:1420:2104 length=100
HDHHHHHHHHHHHHHHHHHHHHHHGHHHHHHHHHHHHHHFHFFHHHHHHHHEHEHHHHHHHGHHHED?EE=A@BACDDECCE@DB74?############

Output

[Desktop]$  sed 's/SRR[0-9]*.[0-9] //g' test.fq
@HWI-ST915_0064:2:1101:1420:2104/1
GTCTCTTCGCACGCTTTCACTGTGAACGGTTCGGCATCGAGAAGGACGCAGTTCCTCTCCGGCTTGGACCAGTTTCTGGTGGCCACGGCTGCCCCCATCC
+HWI-ST915_0064:2:1101:1420:2104 length=100
HDHHHHHHHHHHHHHHHHHHHHHHGHHHHHHHHHHHHHHFHFFHHHHHHHHEHEHHHHHHHGHHHED?EE=A@BACDDECCE@DB74?############

enter image description here

• 0 views • link
written by lakhujanivijay lakhujanivijay

This would still lead to line 1 header not matching line 3. So as @Wouter said above will cause problems with some programs.

• 0 views • link
written by GenoMax GenoMax

I agree and indeed the "length=100" causes the problem so doesn't match.

• 0 views • link
written by majeedaasim • majeedaasim

ah! not a big task to remove that as well! sed once again my friend; adding sed 's/length=[0-9]*//' :)

sed 's/SRR[0-9]*.[0-9] //g' test.fq | sed 's/length=[0-9]*//'

[Desktop]$ sed 's/SRR[0-9]*.[0-9] //g' test.fq | sed 's/length=[0-9]*//'
@HWI-ST915_0064:2:1101:1420:2104/1
GTCTCTTCGCACGCTTTCACTGTGAACGGTTCGGCATCGAGAAGGACGCAGTTCCTCTCCGGCTTGGACCAGTTTCTGGTGGCCACGGCTGCCCCCATCC
+HWI-ST915_0064:2:1101:1420:2104 
HDHHHHHHHHHHHHHHHHHHHHHHGHHHHHHHHHHHHHHFHFFHHHHHHHHEHEHHHHHHHGHHHED?EE=A@BACDDECCE@DB74?############

Is that /1 an issue? If yes, let me know.

• 0 views • link
written by lakhujanivijay lakhujanivijay

Yes it will be. Line 1 and 3 need to match or line 3 need to be blank (past +) for many tools.

• 0 views • link
written by GenoMax GenoMax

To remove anything after '+' from 3rd line, you can use simple sed one liner:

sed "s/+SRR1188607.*/+/g" in_file.fastq >out_file.fastq
• 0 views • link
updated by GenoMax GenoMax • written by Tm Tm

To properly format code you should select the text you want to format first and then click on the '101` button you see in the editor window.

• 0 views • link
written by GenoMax GenoMax

thanks toralmanvar it works But what if the SRR... is present in the the beginning of the first line as

SRR1188607.1@HWI-ST915_0064:2:1101:1498:2108/1

And I want to get rid of these SRR IDs to begin the description line with @HWI- onwards

• 0 views • link
written by majeedaasim • majeedaasim

Dump the reads with -F option.

fastq-dump -F --split-files SRR1188607

and then add 1: and 2: to the read headers by using reformat.sh from BBMap suite.

 reformat.sh in1=SRR1188607_1.fastq in2=SRR1188607_2.fastq out1=R1.fq out2=R2.fq addcolon=t
• 0 views • link
written by GenoMax GenoMax

To replace SRR1188607.1 from the begining, you can use:

sed "s/SRR1188607.1@HWI-ST915/@HWI-ST915/g" in_file.fastq >out_file.fastq

Here SRR1188607.1@HWI-ST915 part will be substituted by @HWI-ST915 in your first header line.

• 0 views • link
written by Tm Tm

It tried it, but the description was same and there was no impact.

Also one point I shall mention is that the change shall take place in all reads and not in a single read as the code you mentioned reflects.

Thanks

• 0 views • link
written by majeedaasim • majeedaasim

Use the method I had posted above.

• 0 views • link
written by GenoMax GenoMax 
$ sed 's/^\(\+\).*\(HWI.*\)/\1\2/g' test.fq
• 0 views • link
written by cpad0112 cpad0112

This also leads to line 1 header not matching line 3. So as @Wouter said above will cause problems with some programs.

• 0 views • link
written by GenoMax GenoMax

output (for other read file replace /1 with /2):

$ sed 's/^\(\+\).*\(HWI.*\)\s.*/\1\2\/1/g' test.fq

or

 $ sed 's/\bSRR.\{9\} \b//;s/\b len.*\b/\/1/' test.fq 
@HWI-ST915_0064:2:1101:1420:2104/1
GTCTCTTCGCACGCTTTCACTGTGAACGGTTCGGCATCGAGAAGGACGCAGTTCCTCTCCGGCTTGGACCAGTTTCTGGTGGCCACGGCTGCCCCCATCC
+HWI-ST915_0064:2:1101:1420:2104/1
HDHHHHHHHHHHHHHHHHHHHHHHGHHHHHHHHHHHHHHFHFFHHHHHHHHEHEHHHHHHHGHHHED?EE=A@BACDDECCE@DB74?############

@HWI-ST915_0064:2:1101:1498:2108/1
AAAGATTGCAATGGAGGAGAAAGGGAAGACCCTGCCTGAAGAAATGCAATTGATAAATAAGTTGTTGTCTGAGGAAAAGGGTTCGGAGAGGATGAGAATG
+HWI-ST915_0064:2:1101:1498:2108/1
GFHHHHHHHHHHHHHHHHHHHHHGHGHHHHHHHHHHGHHHHGHHHHHHDHGGHHFHHHHHGEGGFGGFGFHFHHDHHGHHGFGFGHGFHHHHHHHHHHHH

input:

$ cat test.fq 
@HWI-ST915_0064:2:1101:1420:2104/1
GTCTCTTCGCACGCTTTCACTGTGAACGGTTCGGCATCGAGAAGGACGCAGTTCCTCTCCGGCTTGGACCAGTTTCTGGTGGCCACGGCTGCCCCCATCC
+SRR1188607.1 HWI-ST915_0064:2:1101:1420:2104 length=100
HDHHHHHHHHHHHHHHHHHHHHHHGHHHHHHHHHHHHHHFHFFHHHHHHHHEHEHHHHHHHGHHHED?EE=A@BACDDECCE@DB74?############

@HWI-ST915_0064:2:1101:1498:2108/1
AAAGATTGCAATGGAGGAGAAAGGGAAGACCCTGCCTGAAGAAATGCAATTGATAAATAAGTTGTTGTCTGAGGAAAAGGGTTCGGAGAGGATGAGAATG
+SRR1188607.2 HWI-ST915_0064:2:1101:1498:2108 length=100
GFHHHHHHHHHHHHHHHHHHHHHGHGHHHHHHHHHHGHHHHGHHHHHHDHGGHHFHHHHHGEGGFGGFGFHFHHDHHGHHGFGFGHGFHHHHHHHHHHHH
• 0 views • link
written by cpad0112 cpad0112

Log in to answer this question.

More posts like this
  • SRA to FASTQ problem
    written by m.t.lorenc • m.t.lorenc
    Hi, I downloaded RNA-Seq data from NCBI and used `fastq-dump --gzip --skip-technical --readids --read-filter pass --dumpbase --split-3 --clip SRR1043177.sra` to convert into FASTQ. However, it …
  • Problem in paired-end data due to annotation
    written by priya120195 • priya120195
    Running of Prinseq tool for filtering of paired end reads gives singletons file,good file and bad file separately. I got the correct results when I …
  • Error in TRinity
    written by majeedaasim • majeedaasim
    I downloaded the fastq files from EBI ENA and then ran TRinity on them. Following error is shown Error, not recognizing read name formatting: [SRR1188607.1] …
  • Problem in TRim Galore
    written by majeedaasim • majeedaasim
    I have down loaded the SRA file from using fast-dump as fastq-dump -A SRX485643 --skip-technical --readids --dumpbase --split-3 --clip SRR_ID It produced 2 files 1 …
  • MuMRescueLite.py - multi-mapping reads rescue script failing
    written by newscient • newscient
    I am trying to use the script MuMRescueLite.py from http://genome.gsc.riken.jp/osc/english/dataresource/. That's the link for the software: http://genome.gsc.riken.jp/osc/english/software/src/MuMRescueLite_090522.tar.gz What it does more or less: After mapping, …
  • ncbi sra toolkit, how to modify the fastq format? Need filter information
    written by claude.loverdo • claude.loverdo
    I want to use data from NCBI. The classic fatsq format is: @SRR2192406.1 HWI-ST0860_91:8:1101:1925:1972 length=100 GTTTGNACCATCTTGACAGACTTCAAAAATTGGCTGGGGTCTAAATTGTTCCCCAAGCTGCCCGGCCTCCCCTTCATCTCTTGTCAAGATCGGAAGAGCA +SRR2192406.1 HWI-ST0860_91:8:1101:1925:1972 length=100 CBCFF#2ACFHFHJJIJIJIJJJJJJIGIGHHIJJI??D?@FGIJJJJIIGFHGHIIIEIIIFHHFEEE2?>ABCACDDACCCAAC@>@AA8<22922?C @SRR2192406.1 HWI-ST0860_91:8:1101:1925:1972 length=100 TGACAAGAGATGAAGGGGAGGCCGGGCAGCTTGGGGAACAATTTAGACCCCAGCCAATTTTTGAAGTCTGTCAAGATGGTGCAAACAGATCGGAAGAGCG +SRR2192406.1 …