This is a test version of Biostars. For the public version, visit https://www.biostars.org.
sam flag from paired end reads

HI Everyone, This might be a simple question but I am not able to understand it. For paired end reads I see 83,163 and 99,147 as the sam flags.

Looking at the description from here about sam flags When I put it as 83 I get first in pair and when i put it as 147 I get second in pair although in both the cases the read maps to reverse strand.

What do we mean by first in pair and second in pair?

Thanks

bam

2 answers

I think if I see someone mention the "first in pair" flag one more time, all my hair will fall out... Pierre, even though I think you are doing so much right in the Bioinformatic space, you really need to update the explain sam flags page that the OP linked to contain the correct definitions of the flags as set out in the SAM spec, because even though the SAM spec is ambiguous/silly in some areas, definitions should stay consistent. (even if that means consistently bad...)

There is no first in pair. There's "first segment in template" and "last segment in template". enter image description here

How does the SAM spec define what's first? Apparently its "the read that comes first". Yeah. That's not the most helpful description I know, which is exactly why many people interpret it wrong.

But wait, there's a useful example on pages 8-9 (https://samtools.github.io/hts-specs/SAMv1.pdf)

Multi-segment annotation (e.g. a gene with introns) should be 
described with multiple lines in SAM (like a multi-segment read). 
Where there is a clear biological direction (e.g. a gene), the first 
segment (FLAG bit 0x40) is used for the first section (e.g. the 5'' 
end of the gene). Thus a GenBank entry location like 
complement(join(85052..85354, 85441..85621, 86097..86284)) would 
have three lines in SAM with a common QNAME:

                   FLAG      POS    CIGAR     Optional fields
The 5'' fragment    883   (0x373)   86097   188M FI:i:1 TC:i:3
Middle  fragment    819   (0x333)   85441   181M FI:i:2 TC:i:3
The 3'' fragment    947   (0x3B3)   85052   303M FI:i:3 TC:i:3

So as you can see, here the first-in-segment flags are related to the position of the reads with respect to the genome, not anything to do with the sequencing order. However...

99% of the time, these TWO flags dont tell us anything we dont already know. If the reads map, we can use their POS to determine their correct position in the genome. If they don't map, and we have an unstranded protocol, we cant set the correct first-in-segment anyway. If its stranded, and we have one read mapping so we know where the other lies genomically, even though the read doesnt map, then the flag might be useful. However, this still means that 99.999% of the time, these first-in-segment flags, according to their definition in the SAM spec, are redundant. This is why so many people assume them to mean first-in-sequencing and second-in-sequencing, because that would give them at least some value. However, for the little value that does confer, we also get 1000 "I dont understand what these flags mean!?" posts. See here where a guy got both flags at once, which doesnt make sense if its supposed to be sequencing order, but make sense if you use the strict SAM definitions: C: weird SAM flag explanation

Furthermore, if there were 3 or more reads in a fragment that were sequenced, you still couldnt use these flags to confer sequencing order. Theres only a 1 and a 0 we can use, so its clear this data was never meant to be used for an arbitrary integer length. The position-in-sequencing was supposed to be encoded in the FASTA/Q header, which is the QNAME via a /1 or /2 or /3 or whatever - but since that was never part of the spec either, thats why first-in-segment is being misappropriated as first-in-sequencing.

Its a big deal because even if it took me 7 minutes to write this post, i bet right now, 10s of researchers are puzzled over this and stay silent. Thats really dumb. If we want a X-in-sequencing identifier, we should ask for a new official tag, standardize QNAME formats, or something totally different. We should not hack the flag field to mean other stuff - or if we do, and perhaps actually Picard should - we should make it very clear that we are disregarding the official meaning for those bits and using something else more useful. I have no problems with Picard not following the SAM spec. I specifically like how it sorts QNAMEs, as well as other things. I just think it needs to be made clearer on that particular explain-sam-flags page that your explaining how Picard uses sam flags.

@John: Sending github at broadinstitute.org email about changing the tag explanation page wording may produce desired results.

you really need to update the explain sam flags page that the OP linked to contain the correct definitions of the flags

yes , that's history. :-) I use the htsjdk java api which still uses the 'old' names :-)

Ahh, ok :) Makes sense!

in paired end experiment, the sequencer produces two files. Something like sample.R1.fastq.gz and sample.R2.fastq.gz which are the reads on 5' and 3' of the sequenced fragment. They both contain the same number of reads.

First in pair means that the read comes from sample.R1.fastq.gz Second in pair means that the read comes from sample.R2.fastq.gz

First and Second in pair flags have no meaning about the mapping of the read .

HI Pierre,

I understand this but then I think If the read is 147 it's second in pair although its mapped on the reverse strand while 83 would mean read is on reverse strand but it is first in pair. So that's why 83,163 go together and 99,147 go together for a paired end read

First/second in pair refer to the order in which the reads sequenced; this is independent from alignment: even unmapped reads are first or second in pair.

Log in to answer this question.