This is a test version of Biostars. For the public version, visit https://www.biostars.org.
VarScan warning and exit after Invalid format for pileup at line 1

I'm trying to variant call using VarScan but after sorting the BAM files and creating mpileup files using samtools I get this error using VarScan

Im a novice so any help would be greatly appreciated

java -jar ~/workspace/bioinformaticstools/varscan/varscan-master/VarScan.v2.4.3.jar mpileup2snp /mnt/z/Nathan/WESsamples/unsolved/X366.mpileup >X366.VarScan.snp

Only SNPs will be reported
Warning: No p-value threshold provided, so p-values will not be calculated
Min coverage:   8
Min reads2:     2
Min var freq:   0.2
Min avg qual:   15
P-value thresh: 0.01

Reading input from /mnt/z/Nathan/WESsamples/unsolved/X366.mpileup

Warning: Line ignored: Invalid format for pileup at line 1

╝     �═ BC╗ �╔�B�BCF╝�╗  1 10 11 12 13 14 15 16 17 18 19 2 20 21 22 3 4 5 6 7 8 9 MT X Y GL000192.1 GL000225.1 GL000194.1 GL000193.1 GL000200.1 GL000222.1 GL000212.1 GL000195.1 GL000223.1 GL000224. GL000219.1 GL000205.1 GL000215.1 GL000216.1 GL000217.1 GL000199.1 GL000211.1 GL000213.1 GL000220.1 GL000218.1 GL
000209.1 GL000221.1 GL000214.1 GL000228.1 GL000227.1 GL000191.1 GL000208.1 GL000198.1 GL000204.1 GL000233.1 GL000237.1 GL000230.1 GL000242.1 GL000243.1 GL000241.1 GL000236.1 GL000240.1 GL000206.1 GL000232.1 GL000234.1 GL000202.1 GL000238.1 GL000244.1 GL000248.1 GL000196.1 GL000249.1 GL000246.1 GL000203.1 GL000197.1 GL000245.1 GL000247.1 GL000201.1 GL000235.1 GL000239.1 GL000210.1 GL000231. GL000229.1 GL000226.1 GL000207.1 A   /mnt/z/NathanHaffordTear/WESsamples/PPCDunsolved/X366.sorted.ba �
GL000219.1 GL000205.1 GL000215.1 GL000216.1 GL000217.1 GL000199.1 GL000211.1 GL000213.1 GL000220.1 GL                                                                                  ##samtoolsVersion=0.1.19-96b5f2294a
wes software error variant calling
• 3,950 views • link
updated by Biostar 380 • | written by n.tear 8

Please use the formatting bar (especially the code option) to present your post better. You can use backticks for inline code (`text` becomes text), or select a chunk of text and use the highlighted button to format it as a code block. I've done it for you this time.
code_formatting

• 0 views • link
written by Ram 4.5K

Hi RamRS. Sorry, will do thanks

• 0 views • link
written by n.tear 8

I made the changes already :)

• 0 views • link
written by ATpoint 9.1K

The pileup is corrupted. Please post the entire command that was used to generate it. Did you use nohup?

• 0 views • link
written by ATpoint 9.1K

Hi Thanks for the response. This was the command I used to generate the mpile up file:

samtools mpileup -E -uf /mnt/z/Nathan/ReferenceGenome/Homo_sapiens.GRCh37.dna.primary_assembly.fa /mnt/z/NathanHaffordTear/WESsamples/unsolved/X366.sorted.bam > X366.mpileup

I was following instruction from this paper: https://www.biorxiv.org/content/biorxiv/early/2017/10/11/201145.full.pdf

• 0 views • link
written by n.tear 8

Is that the full command? No nohup or anything, or the command embedded in a wrapper function?

• 0 views • link
written by ATpoint 9.1K

Yes thats the full command. Sorry im not familiar with nohup or embedding in a wrapper

• 0 views • link
written by n.tear 8

The thing is that your pileup looks like it was contaminated by some binary information, maybe a mix of stderr and stdout, that is why I was asking. Can you post the output of 

samtools mpileup -E -uf /mnt/z/Nathan/ReferenceGenome/Homo_sapiens.GRCh37.dna.primary_assembly.fa /mnt/z/NathanHaffordTear/WESsamples/unsolved/X366.sorted.bam | head -n 20
• 0 views • link
written by ATpoint 9.1K 
     head -20 X366.mpileup
╝     ═ BC╗ ╔BBCF╝╗  1 10 11 12 13 14 15 16 17 18 19 2 20 21 22 3 4 5 6 7 8 9 MT X Y GL000192.1 GL000225.1 GL000194.1 GL000193.1 GL000200.1 GL000222.1 GL000212.1 GL000195.1 GL000223.1 GL000224.1 GL000219.1 GL000205.1 GL000215.1 GL000216.1 GL000217.1 GL000199.1 GL000211.1 GL000213.1 GL000220.1 GL000218.1 GL000209.1 GL000221.1 GL000214.1 GL000228.1 GL000227.1 GL000191.1 GL000208.1 GL000198.1 GL000204.1 GL000233.1 GL000237.1 GL000230.1 GL000242.1 GL000243.1 GL000241.1 GL000236.1 GL000240.1 GL000206.1 GL000232.1 GL000234.1 GL000202.1 GL000238.1 GL000244.1 GL000248.1 GL000196.1 GL000249.1 GL000246.1 GL000203.1 GL000197.1 GL000245.1 GL000247.1 GL000201.1 GL000235.1 GL000239.1 GL000210.1 GL000231.1 GL000229.1 GL000226.1 GL000207.1 A   /mnt/z/Nathan/WESsamples/unsolved/X366.sorted.bam                                                                                                                                                                                   ##samtoolsVersion=0.1.19-96b5f2294a##reference=file:///mnt/z/NathanHaffordTear/ReferenceGenome/Homo_sapiens.GRCh37.dna.primary_assembly.fa##contig=<ID=1,length=249250621>##contig=<ID=10,length=135534747>
    ##contig=<ID=11,length=135006516>
    ##contig=<ID=12,length=133851895>
    ##contig=<ID=13,length=115169878>
    ##contig=<ID=14,length=107349540>
    ##contig=<ID=15,length=102531392>
    ##contig=<ID=16,length=90354753>
    ##contig=<ID=17,length=81195210>
    ##contig=<ID=18,length=78077248>
    ##contig=<ID=19,length=59128983>
    ##contig=<ID=2,length=243199373>
    ##contig=<ID=20,length=63025520>
    ##contig=<ID=21,length=48129895>
    ##contig=<ID=22,length=51304566>
    ##contig=<ID=3,length=198022430>
    ##contig=<ID=4,length=191154276>
    ##contig=<ID=5,length=180915260>

I also get this warning when running mpileup with samtools. should I be using bcftools?

[warning] samtools mpileup option `u` is functional, but deprecated. Please switch to using bcftools mpileup in future.
• 0 views • link
written by n.tear 8

I do not know what this is. A mpileup file looks like this:

chr1    3000238 a   1   ^K. A
chr1    3000239 t   1   .   A
chr1    3000240 t   1   .   A
chr1    3000241 t   1   .   A
chr1    3000242 c   1   .   A
chr1    3000243 a   1   .   E
chr1    3000244 t   1   .   E
chr1    3000245 c   1   .   E
chr1    3000246 c   1   .   E
chr1    3000247 a   1   .   E

Do you run this directly via the command line?

• 0 views • link
written by ATpoint 9.1K

okay thanks yes I am running from a ubuntu bash shell on windows. perhaps I need to QC my bam files to check they are correct

• 0 views • link
written by n.tear 8

Looks like a partial VCF file. Maybe a BCF file?

• 0 views • link
written by Ram 4.5K

You should also add the samtools command that you used to generate the pileup, because from the error message it seems like the problem lies there.

• 0 views • link
written by Ram 4.5K

0 answers

No answers yet.

Log in to answer this question.

More posts like this