Entering edit mode
5.2 years ago
Learner
▴
280
I am trying to get ride of PCR duplicate using PICARD. I do it like the following as described in their manual
java -jar picard.jar EstimateLibraryComplexity I=sequence1.sorted.bam O=est_lib_complex_metrics.txt
I am getting the following error, is someone knows what is going on ?
Runtime.totalMemory()=1887436800
To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp
Exception in thread "main" htsjdk.samtools.SAMFormatException: SAM validation error: ERROR: Record 44339180, Read name HWI-ST1378:127:C2J2YACXX:5:1108:15196:95302, MAPQ should be 0 for unmapped read.
at htsjdk.samtools.SAMUtils.processValidationErrors(SAMUtils.java:455)
at htsjdk.samtools.BAMFileReader$BAMFileIterator.advance(BAMFileReader.java:821)
at htsjdk.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:806)
at htsjdk.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:774)
at htsjdk.samtools.SamReader$AssertingIterator.next(SamReader.java:569)
at htsjdk.samtools.SamReader$AssertingIterator.next(SamReader.java:548)
at picard.sam.markduplicates.EstimateLibraryComplexity.doWork(EstimateLibraryComplexity.java:471)
at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:295)
at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:103)
at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:113)
If I add the VALIDATION_STRINGENCY=LENIENT like below
java -jar picard.jar EstimateLibraryComplexity VALIDATION_STRINGENCY=LENIENT I=sequence1.sorted.bam O=est_lib_complex_metrics.txt
I get the following error messages but it finish the job
Ignoring SAM validation error: ERROR: Record 44339180, Read name HWI-ST1378:127:C2J2YACXX:5:1108:15196:95302, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339181, Read name HWI-ST1378:127:C2J2YACXX:5:1214:21260:56139, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339182, Read name HWI-ST1378:127:C2J2YACXX:5:2205:18720:40578, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339184, Read name HWI-ST1378:127:C2J2YACXX:5:2214:12304:94466, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339186, Read name HWI-ST1378:127:C2J2YACXX:5:2109:17696:20808, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339187, Read name HWI-ST1378:127:C2J2YACXX:5:1103:7526:33107, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339188, Read name HWI-ST1378:127:C2J2YACXX:5:1303:15530:2372, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339189, Read name HWI-ST1378:127:C2J2YACXX:5:2203:11558:1994, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339190, Read name HWI-ST1378:127:C2J2YACXX:5:2210:18305:9518, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339191, Read name HWI-ST1378:127:C2J2YACXX:5:1206:4353:92707, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339192, Read name HWI-ST1378:127:C2J2YACXX:5:2111:21303:87709, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390590, Read name HWI-ST1378:127:C2J2YACXX:5:2101:20229:70964, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390592, Read name HWI-ST1378:127:C2J2YACXX:5:2205:5736:97032, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390593, Read name HWI-ST1378:127:C2J2YACXX:5:2209:6851:4809, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390594, Read name HWI-ST1378:127:C2J2YACXX:5:1116:8194:94777, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390595, Read name HWI-ST1378:127:C2J2YACXX:5:2316:12700:76512, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390596, Read name HWI-ST1378:127:C2J2YACXX:5:1103:16407:6018, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390598, Read name HWI-ST1378:127:C2J2YACXX:5:1215:2282:83556, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390599, Read name HWI-ST1378:127:C2J2YACXX:5:2208:7349:61970, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390600, Read name HWI-ST1378:127:C2J2YACXX:5:1313:4872:97245, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390602, Read name HWI-ST1378:127:C2J2YACXX:5:2214:11853:31670, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390603, Read name HWI-ST1378:127:C2J2YACXX:5:1112:9493:82028, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390604, Read name HWI-ST1378:127:C2J2YACXX:5:1210:3216:68622, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390606, Read name HWI-ST1378:127:C2J2YACXX:5:1112:7941:31706, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390607, Read name HWI-ST1378:127:C2J2YACXX:5:1201:7057:59728, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390608, Read name HWI-ST1378:127:C2J2YACXX:5:1201:7080:59728, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390609, Read name HWI-ST1378:127:C2J2YACXX:5:2111:12327:45759, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390610, Read name HWI-ST1378:127:C2J2YACXX:5:1308:4332:33912, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390611, Read name HWI-ST1378:127:C2J2YACXX:5:1312:13789:4596, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390612, Read name HWI-ST1378:127:C2J2YACXX:5:1314:15553:45449, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390613, Read name HWI-ST1378:127:C2J2YACXX:5:2113:2989:79565, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390614, Read name HWI-ST1378:127:C2J2YACXX:5:2209:14800:37450, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390615, Read name HWI-ST1378:127:C2J2YACXX:5:2113:2392:94812, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390616, Read name HWI-ST1378:127:C2J2YACXX:5:2214:1479:90858, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390617, Read name HWI-ST1378:127:C2J2YACXX:5:1209:19164:87879, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390618, Read name HWI-ST1378:127:C2J2YACXX:5:2115:10395:18039, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390619, Read name HWI-ST1378:127:C2J2YACXX:5:2114:16346:76076, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390620, Read name HWI-ST1378:127:C2J2YACXX:5:2304:6831:11971, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390621, Read name HWI-ST1378:127:C2J2YACXX:5:2310:10863:22708, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390622, Read name HWI-ST1378:127:C2J2YACXX:5:1306:20053:21104, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390623, Read name HWI-ST1378:127:C2J2YACXX:5:2211:19830:4187, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390624, Read name HWI-ST1378:127:C2J2YACXX:5:1308:18006:37334, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390625, Read name HWI-ST1378:127:C2J2YACXX:5:2116:9877:5216, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390626, Read name HWI-ST1378:127:C2J2YACXX:5:2116:1719:78990, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390627, Read name HWI-ST1378:127:C2J2YACXX:5:2106:8495:44417, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390628, Read name HWI-ST1378:127:C2J2YACXX:5:1312:9818:71623, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390629, Read name HWI-ST1378:127:C2J2YACXX:5:2101:3401:73465, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390630, Read name HWI-ST1378:127:C2J2YACXX:5:2110:19142:40316, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390631, Read name HWI-ST1378:127:C2J2YACXX:5:2206:1812:54811, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390632, Read name HWI-ST1378:127:C2J2YACXX:5:1116:13054:43638, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390633, Read name HWI-ST1378:127:C2J2YACXX:5:2213:15964:80305, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390634, Read name HWI-ST1378:127:C2J2YACXX:5:2302:8951:70568, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390635, Read name HWI-ST1378:127:C2J2YACXX:5:2108:8159:89940, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390636, Read name HWI-ST1378:127:C2J2YACXX:5:2201:8615:36693, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390637, Read name HWI-ST1378:127:C2J2YACXX:5:1315:9660:11813, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390638, Read name HWI-ST1378:127:C2J2YACXX:5:1212:17412:54925, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390639, Read name HWI-ST1378:127:C2J2YACXX:5:1301:15861:9056, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390640, Read name HWI-ST1378:127:C2J2YACXX:5:1113:13691:48057, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390641, Read name HWI-ST1378:127:C2J2YACXX:5:2207:15182:75393, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390642, Read name HWI-ST1378:127:C2J2YACXX:5:2104:3573:37696, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390643, Read name HWI-ST1378:127:C2J2YACXX:5:2208:12584:47416, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390644, Read name HWI-ST1378:127:C2J2YACXX:5:2102:5171:80898, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390645, Read name HWI-ST1378:127:C2J2YACXX:5:1208:9579:43976, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390646, Read name HWI-ST1378:127:C2J2YACXX:5:1212:6444:12059, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390647, Read name HWI-ST1378:127:C2J2YACXX:5:2113:3697:11779, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390648, Read name HWI-ST1378:127:C2J2YACXX:5:2203:21309:56963, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390649, Read name HWI-ST1378:127:C2J2YACXX:5:2311:20377:98121, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390650, Read name HWI-ST1378:127:C2J2YACXX:5:2216:5939:12495, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390651, Read name HWI-ST1378:127:C2J2YACXX:5:2116:9115:91024, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390652, Read name HWI-ST1378:127:C2J2YACXX:5:1110:3264:79297, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390653, Read name HWI-ST1378:127:C2J2YACXX:5:1303:2798:23101, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390654, Read name HWI-ST1378:127:C2J2YACXX:5:1103:3424:50387, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390655, Read name HWI-ST1378:127:C2J2YACXX:5:2108:14095:82979, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390656, Read name HWI-ST1378:127:C2J2YACXX:5:1103:18252:45369, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390657, Read name HWI-ST1378:127:C2J2YACXX:5:1212:8499:38247, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390658, Read name HWI-ST1378:127:C2J2YACXX:5:1115:13398:53868, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390659, Read name HWI-ST1378:127:C2J2YACXX:5:1312:20072:77452, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390660, Read name HWI-ST1378:127:C2J2YACXX:5:2311:12219:51428, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390661, Read name HWI-ST1378:127:C2J2YACXX:5:2314:9410:77319, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390662, Read name HWI-ST1378:127:C2J2YACXX:5:2108:15835:89431, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44423706, Read name HWI-ST1378:127:C2J2YACXX:5:1115:18269:40664, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44557527, Read name HWI-ST1378:127:C2J2YACXX:5:1316:19319:47895, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44584824, Read name HWI-ST1378:127:C2J2YACXX:5:1316:1754:72960, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44591461, Read name HWI-ST1378:127:C2J2YACXX:5:2108:1958:64473, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44591462, Read name HWI-ST1378:127:C2J2YACXX:5:1210:8430:24166, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44592607, Read name HWI-ST1378:127:C2J2YACXX:5:1101:6521:11468, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44592609, Read name HWI-ST1378:127:C2J2YACXX:5:1110:18181:36652, MAPQ should be 0 for unmapped read.
basically It ignores some unmapped reads that I have in the bam file ! I still do not know which way is the best if I remove the unmapped with Samtools or above
Here is the critical message:
See solutions at Final Solution For "Mapq Should Be 0 For Unmapped Read."
@h.mon I see, I liked your answer, I am waiting for others to see if they propose something different
If you are not able to make this work. You could use
clumpify.shfrom BBMap instead with raw data and get rid of all duplicates before alignments. See: A: Introducing Clumpify: Create 30% Smaller, Faster Gzipped Fastq Files@genomax thanks a lot, I followed what h.man posted , it seems by removing those unmapped bad boys, it will run the Picard !!! however, I am just wondering if It is a good way to do
@Pierre Lindenbaum how to add the VALIDATION_STRINGENY? basically I did
samtools view -bF 4 sequence1.sorted.bam > sequence1sortedfiltered.bamDo you think it is a wrong way to deal with it? I can test the Validation_Stringeny if you tell me how to do that
it's an option for picard, not samtools.
@Pierre Lindenbaum Yes, I showed it above, is there anyway to get those reads or understand them ?
you don't have to understand those reads. It's a bug in bwa which set a mapping quality to the unmapped reads.