This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Picard PCR duplicate gives an error

I am trying to get ride of PCR duplicate using PICARD. I do it like the following as described in their manual 

java -jar picard.jar EstimateLibraryComplexity I=sequence1.sorted.bam O=est_lib_complex_metrics.txt

I am getting the following error, is someone knows what is going on ? 

Runtime.totalMemory()=1887436800
To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp
Exception in thread "main" htsjdk.samtools.SAMFormatException: SAM validation error: ERROR: Record 44339180, Read name HWI-ST1378:127:C2J2YACXX:5:1108:15196:95302, MAPQ should be 0 for unmapped read.
    at htsjdk.samtools.SAMUtils.processValidationErrors(SAMUtils.java:455)
    at htsjdk.samtools.BAMFileReader$BAMFileIterator.advance(BAMFileReader.java:821)
    at htsjdk.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:806)
    at htsjdk.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:774)
    at htsjdk.samtools.SamReader$AssertingIterator.next(SamReader.java:569)
    at htsjdk.samtools.SamReader$AssertingIterator.next(SamReader.java:548)
    at picard.sam.markduplicates.EstimateLibraryComplexity.doWork(EstimateLibraryComplexity.java:471)
    at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:295)
    at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:103)
    at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:113)

If I add the VALIDATION_STRINGENCY=LENIENT like below 

java -jar picard.jar EstimateLibraryComplexity VALIDATION_STRINGENCY=LENIENT I=sequence1.sorted.bam O=est_lib_complex_metrics.txt

I get the following error messages but it finish the job 

Ignoring SAM validation error: ERROR: Record 44339180, Read name HWI-ST1378:127:C2J2YACXX:5:1108:15196:95302, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339181, Read name HWI-ST1378:127:C2J2YACXX:5:1214:21260:56139, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339182, Read name HWI-ST1378:127:C2J2YACXX:5:2205:18720:40578, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339184, Read name HWI-ST1378:127:C2J2YACXX:5:2214:12304:94466, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339186, Read name HWI-ST1378:127:C2J2YACXX:5:2109:17696:20808, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339187, Read name HWI-ST1378:127:C2J2YACXX:5:1103:7526:33107, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339188, Read name HWI-ST1378:127:C2J2YACXX:5:1303:15530:2372, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339189, Read name HWI-ST1378:127:C2J2YACXX:5:2203:11558:1994, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339190, Read name HWI-ST1378:127:C2J2YACXX:5:2210:18305:9518, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339191, Read name HWI-ST1378:127:C2J2YACXX:5:1206:4353:92707, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44339192, Read name HWI-ST1378:127:C2J2YACXX:5:2111:21303:87709, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390590, Read name HWI-ST1378:127:C2J2YACXX:5:2101:20229:70964, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390592, Read name HWI-ST1378:127:C2J2YACXX:5:2205:5736:97032, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390593, Read name HWI-ST1378:127:C2J2YACXX:5:2209:6851:4809, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390594, Read name HWI-ST1378:127:C2J2YACXX:5:1116:8194:94777, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390595, Read name HWI-ST1378:127:C2J2YACXX:5:2316:12700:76512, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390596, Read name HWI-ST1378:127:C2J2YACXX:5:1103:16407:6018, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390598, Read name HWI-ST1378:127:C2J2YACXX:5:1215:2282:83556, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390599, Read name HWI-ST1378:127:C2J2YACXX:5:2208:7349:61970, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390600, Read name HWI-ST1378:127:C2J2YACXX:5:1313:4872:97245, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390602, Read name HWI-ST1378:127:C2J2YACXX:5:2214:11853:31670, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390603, Read name HWI-ST1378:127:C2J2YACXX:5:1112:9493:82028, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390604, Read name HWI-ST1378:127:C2J2YACXX:5:1210:3216:68622, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390606, Read name HWI-ST1378:127:C2J2YACXX:5:1112:7941:31706, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390607, Read name HWI-ST1378:127:C2J2YACXX:5:1201:7057:59728, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390608, Read name HWI-ST1378:127:C2J2YACXX:5:1201:7080:59728, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390609, Read name HWI-ST1378:127:C2J2YACXX:5:2111:12327:45759, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390610, Read name HWI-ST1378:127:C2J2YACXX:5:1308:4332:33912, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390611, Read name HWI-ST1378:127:C2J2YACXX:5:1312:13789:4596, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390612, Read name HWI-ST1378:127:C2J2YACXX:5:1314:15553:45449, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390613, Read name HWI-ST1378:127:C2J2YACXX:5:2113:2989:79565, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390614, Read name HWI-ST1378:127:C2J2YACXX:5:2209:14800:37450, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390615, Read name HWI-ST1378:127:C2J2YACXX:5:2113:2392:94812, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390616, Read name HWI-ST1378:127:C2J2YACXX:5:2214:1479:90858, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390617, Read name HWI-ST1378:127:C2J2YACXX:5:1209:19164:87879, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390618, Read name HWI-ST1378:127:C2J2YACXX:5:2115:10395:18039, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390619, Read name HWI-ST1378:127:C2J2YACXX:5:2114:16346:76076, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390620, Read name HWI-ST1378:127:C2J2YACXX:5:2304:6831:11971, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390621, Read name HWI-ST1378:127:C2J2YACXX:5:2310:10863:22708, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390622, Read name HWI-ST1378:127:C2J2YACXX:5:1306:20053:21104, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390623, Read name HWI-ST1378:127:C2J2YACXX:5:2211:19830:4187, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390624, Read name HWI-ST1378:127:C2J2YACXX:5:1308:18006:37334, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390625, Read name HWI-ST1378:127:C2J2YACXX:5:2116:9877:5216, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390626, Read name HWI-ST1378:127:C2J2YACXX:5:2116:1719:78990, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390627, Read name HWI-ST1378:127:C2J2YACXX:5:2106:8495:44417, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390628, Read name HWI-ST1378:127:C2J2YACXX:5:1312:9818:71623, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390629, Read name HWI-ST1378:127:C2J2YACXX:5:2101:3401:73465, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390630, Read name HWI-ST1378:127:C2J2YACXX:5:2110:19142:40316, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390631, Read name HWI-ST1378:127:C2J2YACXX:5:2206:1812:54811, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390632, Read name HWI-ST1378:127:C2J2YACXX:5:1116:13054:43638, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390633, Read name HWI-ST1378:127:C2J2YACXX:5:2213:15964:80305, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390634, Read name HWI-ST1378:127:C2J2YACXX:5:2302:8951:70568, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390635, Read name HWI-ST1378:127:C2J2YACXX:5:2108:8159:89940, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390636, Read name HWI-ST1378:127:C2J2YACXX:5:2201:8615:36693, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390637, Read name HWI-ST1378:127:C2J2YACXX:5:1315:9660:11813, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390638, Read name HWI-ST1378:127:C2J2YACXX:5:1212:17412:54925, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390639, Read name HWI-ST1378:127:C2J2YACXX:5:1301:15861:9056, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390640, Read name HWI-ST1378:127:C2J2YACXX:5:1113:13691:48057, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390641, Read name HWI-ST1378:127:C2J2YACXX:5:2207:15182:75393, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390642, Read name HWI-ST1378:127:C2J2YACXX:5:2104:3573:37696, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390643, Read name HWI-ST1378:127:C2J2YACXX:5:2208:12584:47416, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390644, Read name HWI-ST1378:127:C2J2YACXX:5:2102:5171:80898, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390645, Read name HWI-ST1378:127:C2J2YACXX:5:1208:9579:43976, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390646, Read name HWI-ST1378:127:C2J2YACXX:5:1212:6444:12059, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390647, Read name HWI-ST1378:127:C2J2YACXX:5:2113:3697:11779, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390648, Read name HWI-ST1378:127:C2J2YACXX:5:2203:21309:56963, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390649, Read name HWI-ST1378:127:C2J2YACXX:5:2311:20377:98121, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390650, Read name HWI-ST1378:127:C2J2YACXX:5:2216:5939:12495, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390651, Read name HWI-ST1378:127:C2J2YACXX:5:2116:9115:91024, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390652, Read name HWI-ST1378:127:C2J2YACXX:5:1110:3264:79297, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390653, Read name HWI-ST1378:127:C2J2YACXX:5:1303:2798:23101, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390654, Read name HWI-ST1378:127:C2J2YACXX:5:1103:3424:50387, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390655, Read name HWI-ST1378:127:C2J2YACXX:5:2108:14095:82979, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390656, Read name HWI-ST1378:127:C2J2YACXX:5:1103:18252:45369, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390657, Read name HWI-ST1378:127:C2J2YACXX:5:1212:8499:38247, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390658, Read name HWI-ST1378:127:C2J2YACXX:5:1115:13398:53868, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390659, Read name HWI-ST1378:127:C2J2YACXX:5:1312:20072:77452, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390660, Read name HWI-ST1378:127:C2J2YACXX:5:2311:12219:51428, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390661, Read name HWI-ST1378:127:C2J2YACXX:5:2314:9410:77319, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44390662, Read name HWI-ST1378:127:C2J2YACXX:5:2108:15835:89431, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44423706, Read name HWI-ST1378:127:C2J2YACXX:5:1115:18269:40664, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44557527, Read name HWI-ST1378:127:C2J2YACXX:5:1316:19319:47895, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44584824, Read name HWI-ST1378:127:C2J2YACXX:5:1316:1754:72960, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44591461, Read name HWI-ST1378:127:C2J2YACXX:5:2108:1958:64473, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44591462, Read name HWI-ST1378:127:C2J2YACXX:5:1210:8430:24166, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44592607, Read name HWI-ST1378:127:C2J2YACXX:5:1101:6521:11468, MAPQ should be 0 for unmapped read.
Ignoring SAM validation error: ERROR: Record 44592609, Read name HWI-ST1378:127:C2J2YACXX:5:1110:18181:36652, MAPQ should be 0 for unmapped read.

basically It ignores some unmapped reads that I have in the bam file ! I still do not know which way is the best if I remove the unmapped with Samtools or above

genomics
• 2,354 views • link
written by Learner Learner

SAM validation error: ERROR: Record 44339180, Read name HWI-ST1378:127:C2J2YACXX:5:1108:15196:95302, MAPQ should be 0 for unmapped read.

Therefore, output of samtools view your.bam | grep 'HWI-ST1378:127:C2J2YACXX:5:1108:15196:95302' please or alternatively see if you can set validation strigency to lenient, think it is something like VALIDATION_STRINGENY=lenient

• 0 views • link
written by ATpoint ATpoint

@ATpoint Your comment gives me this 

samtools view sequence1.sorted.bam | grep 'HWI-ST1378:127:C2J2YACXX:5:1108:15196:95302'

HWI-ST1378:127:C2J2YACXX:5:1108:15196:95302 117 chr9_gl000201_random    36145   0   *   =   36145   0   NGATAGACCTGTGATCCATCGTGATGTCTTATTTAAGGGGAACGTGTGGGCTATTTAGGCTTTATGGCCCTGAAGTAGGAACCAGATGTCGGATACAGTT    DDECDCDDDDDDDDDDBD@DEEEECFFFFFFHHHHHIIHGHFGJIJJJIEJJIJIJJJIJJJIJJJJJJJJJJJJJJJJJJIJJJIJHHHHHFFFFFCCC
HWI-ST1378:127:C2J2YACXX:5:1108:15196:95302 157 chr9_gl000201_random    36145   37  100M    =   36145   0   GATGGATCACAGGTCTATCACCCTATTAACCACTCACGGGAGCTCTCCATGCATTTGGTATTTTCGTCTGGGGGGTATGCACGCGATAGCATTGCGAGAC    CDDDDEECCCCCDDDEDDBABDDEECDDCDCDDDBADDDDDDBCEEEFFDFEDFHHHHHIJJJIIHJIJJJJJJJJJJIIIJJJJJJHHHHHFFFFFCCC    XT:A:U  NM:i:SM:i:37    AM:i:0  X0:i:1  X1:i:0  XM:i:2  XO:i:0  XG:i:0  MD:Z:3C72G23
• 1 views • link
written by Learner Learner

@h.mon I see, I liked your answer, I am waiting for others to see if they propose something different

• 1 views • link
written by Learner Learner

@genomax thanks a lot, I followed what h.man posted , it seems by removing those unmapped bad boys, it will run the Picard !!! however, I am just wondering if It is a good way to do

• 1 views • link
written by Learner Learner

@Pierre Lindenbaum how to add the VALIDATION_STRINGENY? basically I did samtools view -bF 4 sequence1.sorted.bam > sequence1sortedfiltered.bam

Do you think it is a wrong way to deal with it? I can test the Validation_Stringeny if you tell me how to do that

• 1 views • link
written by Learner Learner

@Pierre Lindenbaum Yes, I showed it above, is there anyway to get those reads or understand them ?

• 1 views • link
written by Learner Learner

you don't have to understand those reads. It's a bug in bwa which set a mapping quality to the unmapped reads.

• 1 views • link
written by Pierre Lindenbaum Pierre Lindenbaum

0 answers

No answers yet.

Log in to answer this question.

More posts like this