Yeah, so the quality scores are definitely encoded in Sanger format, as the ascii characters are out of range of the phred64 encoding, but they only range from 0-10, which means the lowest error probability for any base is 10%. This can't be right. I'm wondering where they could have been screwed up and if there's any way of recovering the actual quality scores. The authors of the original paper said they discarded reads with average Q below 10 (or maybe 20?). If these quality scores were correct then they would have discarded ALL the data.
messed up SRA quality scores
I'm having an issue with sequences I download from the SRA having apparently incorrect base quality scores.
I downloaded the NCBI SRA run SRR4181534 using sratoolkit 2.11.3 by:
fasterq-dump SRR4181534
But the quality scores must be encoded incorrectly. I ran fastp and none of the reads passed the default filters. The report says the phred scaled scores range from 0-10 (ascii characters 33-43), and there is a very narrow range of quality scores in the actual sequence, so fastp isn't wrong. As an example:
@SRR4181534.1 1 length=101
NACTGTTTGATACTAATAGGTAATTGATTACTTGAGTTAATCACCAAACTCTATCCCTTACTCTGTAANNNTTTTAATGAACTTATTCTTCAATGCAAGCA
+SRR4181534.1 1 length=101
!!!"%%%%''$!%(""(#)'"'&')(!#&"!$)(&!$$!$$'!!%"!'%)'%('!!!#'!!!'')*$#!!!!!!%!!!"!&)&%%$#!!$!$$$$$$$$#!
But I don't believe the original quality scores could all be that bad.
I'm guessing the data were originally encoded as Phred33, but somewhere along the line somebody thought they were Phred64 and converted them (maybe NCBI? maybe during the download process?).
So for example a phred score of 31 + 33 offset = 64. If you incorrectly think the offset is 64 and try to convert, then everything < phred 31 gets truncated to phred 0 (ascii 33). I don't think about this much and I find it all very confusing, but does that make sense? Is there any way to fix this or are the quality scores lost?
1 answer
If I test 5 reads from this SRR# then I get the following.
$ testformat.sh -Xmx2g in=SRR4181534_1.fastq
sanger fastq raw single-ended 101bp
$ testformat.sh -Xmx2g in=SRR4181534_2.fastq
sanger fastq raw single-ended 101bp
That program is from BBMap suite.
Under the Data Access tab original data submitted is available as a BAM file https://trace.ncbi.nlm.nih.gov/Traces/?view=run_browser&acc=SRR4181534&display=data-access You can try to download it from there and see what the data contains.
Looking at a random read the Q scores are all 30 though at SRA site. Is there a chance that you have corrupt data?
Log in to answer this question.
More posts like this
-
Download and convert SRA file to fastq file
written by rescueson •
-
sra toolkit
written by Esra •
-
What is the precise range that Solexa quality scores can take?
written by Aspire
-
Abnormal fastq lines
written by jaqx008
-
Change Phred quality score with DWGSIM
written by marongiu.luigi
-
RSEM ignores quality scores?
written by AW
-
Trimming of 16S reads
written by agata88
-
Filtering sanger_fastq, illumina_fastq files using Fastx-Toolkit
written by bioinfo
-
Fastq-Dump Quality Scores Off Scale
written by Clare
-
Fastq Format Confusion
written by Biomonika (Noolean)
