This is a test version of Biostars. For the public version, visit https://www.biostars.org.
SAMFormatException: Did not inflate expected amount error

I am facing an error while running GATK Basecalibrator.

[1 July 2021 at 5:00:42 PM IST] org.broadinstitute.hellbender.tools.walkers.bqsr.BaseRecalibrator done. Elapsed time: 35.00 minutes.
Runtime.totalMemory()=791674880
htsjdk.samtools.SAMFormatException: Did not inflate expected amount
    at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:147)
    at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:96)
    at htsjdk.samtools.util.BlockCompressedInputStream.inflateBlock(BlockCompressedInputStream.java:550)
    at htsjdk.samtools.util.BlockCompressedInputStream.processNextBlock(BlockCompressedInputStream.java:532)
    at htsjdk.samtools.util.BlockCompressedInputStream.nextBlock(BlockCompressedInputStream.java:468)
    at htsjdk.samtools.util.BlockCompressedInputStream.readBlock(BlockCompressedInputStream.java:458)
    at htsjdk.samtools.util.BlockCompressedInputStream.available(BlockCompressedInputStream.java:196)
    at htsjdk.samtools.util.BlockCompressedInputStream.read(BlockCompressedInputStream.java:241)
    at htsjdk.tribble.readers.TabixReader.readLine(TabixReader.java:215)
    at htsjdk.tribble.readers.TabixReader.access$300(TabixReader.java:48)
    at htsjdk.tribble.readers.TabixReader$IteratorImpl.next(TabixReader.java:434)
    at htsjdk.tribble.readers.TabixIteratorLineReader.readLine(TabixIteratorLineReader.java:46)
    at htsjdk.tribble.TabixFeatureReader$FeatureIterator.readNextRecord(TabixFeatureReader.java:170)
    at htsjdk.tribble.TabixFeatureReader$FeatureIterator.<init>(TabixFeatureReader.java:159)
    at htsjdk.tribble.TabixFeatureReader.query(TabixFeatureReader.java:133)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.refillQueryCache(FeatureDataSource.java:569)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.queryAndPrefetch(FeatureDataSource.java:538)
    at org.broadinstitute.hellbender.engine.FeatureManager.getFeatures(FeatureManager.java:352)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:173)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:125)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:263)
    at org.broadinstitute.hellbender.tools.walkers.bqsr.BaseRecalibrator.apply(BaseRecalibrator.java:189)
    at org.broadinstitute.hellbender.engine.ReadWalker.lambda$traverse$0(ReadWalker.java:96)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
    at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
    at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
    at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
    at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
    at org.broadinstitute.hellbender.engine.ReadWalker.traverse(ReadWalker.java:94)
    at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1058)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:140)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
    at org.broadinstitute.hellbender.Main.main(Main.java:289)

What does this mean? Is it related to memory? My system has 16 GB memory, still it is giving me this error again and again. What to do to get rid of this?

wes

what was the command line ? what is the output of the command file /path/to/file.gz with all your files with a .gz extension ? what is the output of the command gunzip -t /path/to/file.gz with all your files with a .gz extension ?

with this error, I didn tget any output, so there is no question of output in .gz extension. My command line is: java -jar ../../gatk-4.2.0.0/gatk-package-4.2.0.0-local.jar BaseRecalibrator --input sort_Dup_aligned_reads_gatk.bam --known-sites ../../Common_database/GRCh37_latest_dbSNP_all.vcf.gz --output base_recall.table --reference ../../Common_database/GRCh37_latest_genomic.fna

so there is no question of output in .gz

it is a problem with your inputs

htsjdk.samtools.SAMFormatException: Did not inflate expected amount
    at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:147)

I am again facing issue while running gatk HaplotypeCaller.

12:49:43.504 INFO  VectorLoglessPairHMM - Time spent in setup for JNI call : 0.16342299700000001
12:49:43.505 INFO  PairHMM - Total compute time in PairHMM computeLogLikelihoods() : 29.143036857000002
12:49:43.505 INFO  SmithWatermanAligner - Total compute time in java Smith-Waterman : 31.03 sec
12:49:43.505 INFO  HaplotypeCaller - Shutting down engine
[16 July 2021 at 12:49:43 PM IST] org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCaller done. Elapsed time: 3.36 minutes.
Runtime.totalMemory()=1050673152
htsjdk.samtools.SAMFormatException: Did not inflate expected amount
    at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:147)
    at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:96)
    at htsjdk.samtools.util.BlockCompressedInputStream.inflateBlock(BlockCompressedInputStream.java:550)
    at htsjdk.samtools.util.BlockCompressedInputStream.processNextBlock(BlockCompressedInputStream.java:532)
    at htsjdk.samtools.util.BlockCompressedInputStream.nextBlock(BlockCompressedInputStream.java:468)
    at htsjdk.samtools.util.BlockCompressedInputStream.readBlock(BlockCompressedInputStream.java:458)
    at htsjdk.samtools.util.BlockCompressedInputStream.available(BlockCompressedInputStream.java:196)
    at htsjdk.samtools.util.BlockCompressedInputStream.read(BlockCompressedInputStream.java:241)
    at htsjdk.tribble.readers.TabixReader.readLine(TabixReader.java:215)
    at htsjdk.tribble.readers.TabixReader.access$300(TabixReader.java:48)
    at htsjdk.tribble.readers.TabixReader$IteratorImpl.next(TabixReader.java:434)
    at htsjdk.tribble.readers.TabixIteratorLineReader.readLine(TabixIteratorLineReader.java:46)
    at htsjdk.tribble.TabixFeatureReader$FeatureIterator.readNextRecord(TabixFeatureReader.java:170)
    at htsjdk.tribble.TabixFeatureReader$FeatureIterator.<init>(TabixFeatureReader.java:159)
    at htsjdk.tribble.TabixFeatureReader.query(TabixFeatureReader.java:133)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.refillQueryCache(FeatureDataSource.java:569)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.queryAndPrefetch(FeatureDataSource.java:538)
    at org.broadinstitute.hellbender.engine.FeatureManager.getFeatures(FeatureManager.java:352)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:173)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:125)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:240)
    at org.broadinstitute.hellbender.tools.walkers.annotator.VariantOverlapAnnotator.annotateRsID(VariantOverlapAnnotator.java:62)
    at org.broadinstitute.hellbender.tools.walkers.annotator.VariantAnnotatorEngine.annotateContext(VariantAnnotatorEngine.java:305)
    at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCallerGenotypingEngine.makeAnnotatedCall(HaplotypeCallerGenotypingEngine.java:488)
    at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCallerGenotypingEngine.assignGenotypeLikelihoods(HaplotypeCallerGenotypingEngine.java:224)
    at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCallerEngine.callRegion(HaplotypeCallerEngine.java:678)
    at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCaller.apply(HaplotypeCaller.java:273)
    at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.processReadShard(AssemblyRegionWalker.java:200)
    at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.traverse(AssemblyRegionWalker.java:173)
    at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1058)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:140)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
    at org.broadinstitute.hellbender.Main.main(Main.java:289)

Earlier also, I am getting the same issue with BaseRecalibrator which was raised in this thread. Can anyone please tell me whats the error about? I re-installed java but still didnt get any output.

So both of these problems remain unresolved?

You should post in the GATK forum for help for a specific response. Someone may still answer here.

GenoMax both the errors are same, related to java. I just want to know what this error is about?

Pierre already told you above that the error is with your input files. Something appears to be wrong with the compression of those files.

... and I asked you for the output of gunzip -t /path/to/file.gz....

I got this: gzip: no_chr_dbsnp.vcf.gz: invalid compressed data--crc error

gzip: no_chr_dbsnp.vcf.gz: invalid compressed data--length error

Hi, Pierre! I encountered the same error in the step of HaplotypeCaller. Which file do you mean? In my case, my output is vcf WITHOUT .gz. But I still got this error.

Hi, smrutimayipanda. Did you solved the problem? I'm working on WGS analysis of a pedigree of three individuals using GATK 4.2.0.0. I encountered exactly the same error in the step of generating gvcf file from bam file.

It was interesting that everything went on well for the first individual; I got gvcf file containing information of all the chromosomes for him. However, for the other two of the individuals, I encountered the error [htsjdk.samtools.SAMFormatException: Did not inflate expected amount] in processing chr6 and chr11, respectively.

You have met this error twice in BaseRecalibrator and HaplotypeCaller. Have you solved the one in BaseRecalibrator?

No I was not able to solve that. I used another pipeline for variant calling.

I contacted with the GATK team. They asked me to try using jdk-inflater and jdk-deflater, but the problem was not solved yet. Which pipeline you've changed to?

Thanks for the information. I think I 'd better shift to that, too.

0 answers

No answers yet.

Log in to answer this question.