This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Interpretation Of Consensus Sequence From Samtools Pileup

I have generated a consensus sequence using 'pileup' then 'pileup2fq' from samtools. Can anyone tell me exactly what determines whether the resulting sequence is in UPPER or lower case?

An example of the fastq is:

@header
GTTAAGATGAAACATTTACAGGATTTGATTGACGAACCTGATGAtttttcacaacccaat ccatCTtagactagaaaggtaTTTACGGTTGCTaaacattgcgttatgtttaaGACCTCA TGCCAATAGACTGTTTGAATTTTATGAactgtctcctttgggaaacttgttaagtcgtga aastnnnnnnnnnnnnnnncaagggtacttggtcatcagatctaccgcaaaagctCAAGG
+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~r oZMH!!!!!!!!!!!!!!!KZo~~Z~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Thanks.

samtools pileup
• 7,148 views • link
updated by Pablo Marin-Garcia Pablo Marin-Garcia • written by Ian Ian

2 answers

Thanks for viewing, but i found my answer here:

samtools.pl pileup2fq -D100 > var-X.fq

File var-X.fq is in the multi-line FASTQ format. Bases in lowercase are filtered out due to repeats, being close to indels or insufficient/excessive read depth. The consensus file is essential to the estimate of mutation rate. The pileup2fq command applies fewer filters than varFilter and may not give identical results.

• 3 views • link
updated by Istvan Albert Istvan Albert • written by Ian Ian

In this case, I guess you can mark your own answer as the best :-)

• 0 views • link
written by Neilfws Neilfws

I wish i hadn't asked it now....

• 0 views • link
written by Ian Ian

so just to make sure, the lower case letters were either covered but filtered out due to the aforementioned reasons or they were just not covered in any of the reads?

• 0 views • link
written by Doctoroots Doctoroots

From my experience, if they are not at all covered then you do not get any bases in the fastq file.

• 0 views • link
written by Ian Ian

I run the command but the lower case letters are not filtered at all. Why?

• 0 views • link
written by Love • Love

Hi Ian. Just curious. After coverting to FASTQ format, how you end up in estimation of mutation rate (which software/tools do you used)?

• 0 views • link
written by jackuser1979 jackuser1979

The lower case letter are what is called 'soft masking' (bases in low complexity regions like repeats etc.). I don't know if it would be your case but some people provide the reference genome soft masked to the aligners in order to avoid alignments in this regions, but nowadays this is saw as a not good practice and I think that aligners like BWA does not filter out soft masked regions following this biostars answer and similar posts

• 0 views • link
updated by Ram Ram • written by Pablo Marin-Garcia Pablo Marin-Garcia

+1 @Casey: You are right Casey, probably I did not express myself with property. When I was talking about soft-masking I was meaning marking bases to be filtered out. Although the sof-masking rules can be slightly different the concept is the same, as the samtools manual says: "Bases in lowercase are filtered out due to repeats, being close to indels or insufficient/excessive read depth"

• 0 views • link
written by Pablo Marin-Garcia Pablo Marin-Garcia

Log in to answer this question.

More posts like this