This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Difference between samtools "-f 9" and "-f 11"

So ran into an interesting outcome from samtools filtering. I'm attempting to select reads in which one member of the pair is mapped, but the other is not. After searching for the best filtering for that, most people seemed to indicate -f 11 was the correct choice for this. However, that includes "properly paired" which seems like something I don't want. Additionally when I used -f 11 I get nothing coming out. I decided to try -f 9 which removes the proper pairing component and that seemed to work for my needs.

Can anyone explain why the posts I've seen in the past suggest -f 11 and whether or not this has an internal contradiction which yields no reads?

samtools sam bam
• 1,344 views • link
updated by Istvan Albert Istvan Albert • written by Dylan • Dylan

1 answer

9: read must be paired AND mate must be unmapped

11: read must be paired AND mapped in proper pair AND mate must be unmapped (which is weird, if it's in proper pair, read and its mate should be both mapped)

I'm attempting to select reads in which one member of the pair is mapped, but the other is not.

I think you want:

samtools view --expr 'flag.paired && ((flag.unmap && !flag.munmap) || (!flag.unmap && flag.munmap))'  in.bam
• 0 views • link
written by Pierre Lindenbaum Pierre Lindenbaum

(which is weird, if it's in proper pair, read and its mate should be both mapped)

So that was my thought as well, I think the code you listed essentially mirrors doing -f 9 and -f 5 together

• 0 views • link
written by Dylan • Dylan

It is very easy to make nonsensical flags (unfortunately a flaw of the SAM spec itself) ... not to mention the awkward constructs: a primary alignment is an alignment that is not secondary, not supplementary and not unmapped ...

• 0 views • link
written by Istvan Albert Istvan Albert

Log in to answer this question.

More posts like this