I'm having a problem calling SNPs from a sorted bam file with GATK HaplotypeCaller. The file is a merged file of two technical replicates that were each sequenced on different lanes and aligned to the reference genome. When I run Picard ValidateSamFile on the bam file, I got the following error: "ValidateSamFile Value was put into PairInfoMap more than once." I also ran ValidateSamFile on the intermediate sam file for this sample (before its was converted to bam and sorted), and ValidateSamFile did not give me that error message when ran on the sam file. So, it appears that during the step where the sam file was converted into a bam file and sorted with samtools sort, something about read group information was disrupted.
Has anyone encountered this before, or have an ideas as to what would cause this issue? I tried aligning this sample to the reference genome again using the -M tag in bwa-mem (initially I didn't use the -M tag), but this issue wasn't resolved. I haven't been able to find much information about this error, other than suggestions to run Picard AddOrReplaceReadGroups to rename read group information. I'm will likely try this next but I'm going to need to call SNPs on a handful of files like this, so if possible I'd to figure out and fix what the underlying problem is before I resort to replacing the read group information for each sample.
Thank you in advance for your help!
so you have the same A00975:57:HHGM7DRXX:2:2122:1922:36949 at multiple times.
A00975:57:HHGM7DRXX:2:2122:1922:36949 129 A01 947 0 73M78S A02 31442037 0 AAACCCTAAACCCTAAACCCTAAACCCTAAACACTAAACCCTAAACCCGAGACCCTAAACCCTAATACCTAATGCGTATAGCCTAGAGGGTGCACCCTAAAGGCTATACCGGAGGGGGGGGGGCGGGGGGGGGGGGGGGGGGGGGGGGGGG FFFFFFFFF:FFFFFFFFFFFFF:FFFFF::F,:,,,,,:F,F,,,,F,F,,F,,,:FF::,:,,,,,FF,F,,,,FF,,,:,,,,,,,,F,,,,,::,,,,,F,:,:,,,F,,,F,:,,,FF,,F,F::F,FF,::F::,:FFFFF:FF, NM:i:4 MD:Z:32C15T1A14A7 MC:Z:45M106S AS:i:53 XS:i:52SA:Z:A04,11090195,+,115S36M,0,0;
A00975:57:HHGM7DRXX:2:2122:1922:36949 129 A01 947 0 73M78S A02 31442037 0 AAACCCTAAACCCTAAACCCTAAACCCTAAACACTAAACCCTAAACCCGAGACCCTAAACCCTAATACCTAATGCGTATAGCCTAGAGGGTGCACCCTAAAGGCTATACCGGAGGGGGGGGGGCGGGGGGGGGGGGGGGGGGGGGGGGGGG FFFFFFFFF:FFFFFFFFFFFFF:FFFFF::F,:,,,,,:F,F,,,,F,F,,F,,,:FF::,:,,,,,FF,F,,,,FF,,,:,,,,,,,,F,,,,,::,,,,,F,:,:,,,F,,,F,:,,,FF,,F,F::F,FF,::F::,:FFFFF:FF, NM:i:4 MD:Z:32C15T1A14A7 MC:Z:45M106S AS:i:53 XS:i:52SA:Z:A07,18144323,-,5S36M110S,0,0;
the first and the second have the very same sequence, the very same sam flag 129 = read paired (0x1) and second in pair (0x80) ) == it is impossible to find the very same read twice in a BAM file. there is something broken in your upstream workflow.
funilly the first read have the very same sequence (AAACCCTAAACCCTAAACCC(...)GG) are mapped are the very same place but don't have the same SA attributes 'SA:Z:A04,11090195,+,115S36M,0,0;' vs 'SA:Z:A07,18144323,-,5S36M110S,0,0;'
Thanks! This is very confusing. I did not get this error when I ran ValidateSamFile on the very same sample's sam file- does this mean that specifically when I convert from sam to bam with samtools this error is occurring and somehow reads are getting handled incorrectly?
how did you run bwa mem ?
try to find the read "A00975:57:HHGM7DRXX:2:2122:1922:36949" in your fastqs. Check the sequence, check the occurence.
Okay, so I found 4 matches for the read ID "A00975:57:HHGM7DRXX:2:2122:1922:36949" in my raw fastqs: 2 were the sequence starting with GTTT and 2 were the sequence starting with AAAC. The other two sequences were not found.
The basic steps to my pipeline were to:
Here Brapa_v3.0.fasta is the referene genome, Library-17_S17_L001_R1_001.fastq.gz is the forward read file and Library-17_S17_L001_R2_001.fastq.gz is the reverse read file from paired end sequencing. I also repeated this step for a second pair of files of Library-17, L002, which was a technical replicate ran on another flow cell to get enough coverage of the library.
So I have 6 reads with the same read ID in my final bam file. Interestingly, these 4 reads with the same ID that I found in my raw fastqs all came from the Lane 2 fastq files- so the duplicate reads don't appear to be due to the technical replicate. And again, the 5th and 6th duplications of the same read ID were not found in the raw fastq files.
Do you still believe this to be an issue with the pipeline? I'm not sure what to make of 4 duplicate read IDs in the raw fastq files AND 2 additional duplicates that show up in the resulting bam file?
Okay, so I found 4 matches for the read ID "A00975:57:HHGM7DRXX:2:2122:1922:36949" in my raw fastqs: 2 were the sequence starting with GTTT and 2 were the sequence starting with AAAC. The other two sequences were not found.
this is your problem, you're asking ValidateSamFile to do his job. And his job is to check that two reads don't share the same ID. There is a problem with your upstream process that generated the fastq files.
Okay, so this issue is something that was already present when I got these raw fastq files from the sequencing facility and not with my pipeline. That is what you think based on the duplicate reads in the raw fastq files, correct?
Log in to answer this question.
cross-posted to another site: https://gatk.broadinstitute.org/hc/en-us/community/posts/360077152632-Error-ValidateSamFile-Value-was-put-into-PairInfoMap-more-than-once-
this is the first part of the message. What is the second part ? ( https://github.com/samtools/htsjdk/blob/master/src/main/java/htsjdk/samtools/CoordinateSortedPairInfoMap.java#L132 )
Thanks for your reply. Unfortunately, there's is not another part to the error message. After the "ValidateSamFile Value was put into PairInfoMap more than once" there is a "1:" followed by a 31 digit sequence and then things like the date/time and a finished message are output.
please run
what is the output ?
It found 6 matches:
where is the "the-31-digit-sequence" in the first column ??
I didn't copy the first column with that information- I didn't know that sequence is important, but it was there in the first column. Would that help to figure out what the issue is?
It was A00975:57:HHGM7DRXX:2:2122:1922:36949 for each of the 6 rows
this doesn't look like a SAM file; For example the cigar string 73M78S should be in the 6th column while your example it's in the 5th column.