This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Quickest way to filter vcf using bed files

I am filtering vcf files using bed files using vcftools. I have one bed file per vcf (split by chromosome):

for i in "${chroms[@]}"; do vcftools --gzvcf denisovan/chr"$i"_mq25_mapab100.vcf.gz --bed bed/chr"$i"_mask.bed --recode --keep-INFO-all --stdout | gzip -c > filtered/denisovan.filtered."$i".vcf.gz; done

This is proving extremely slow. Is there any tool that is much quicker for doing this?

snp
• 7,129 views • link
written by spiral01 spiral01

Hi, what is the reasoning for using bgzip over gzip? Thanks.

• 0 views • link
written by spiral01 spiral01

1 answer

use gnu parallel :

 (seq 1 10 && echo X && echo Y) | parallel ' vcftools --gzvcf denisovan/chr{}_mq25_mapab100.vcf.gz (...) '
• 0 views • link
written by Pierre Lindenbaum Pierre Lindenbaum

That's extremely useful. Thank you.

• 0 views • link
written by spiral01 spiral01

I have been running the above code as such since yesterday:

(seq 1 22 && echo X) | parallel ' vcftools --gzvcf denisovan/chr{}_mq25_mapab100.vcf.gz --bed bed/chr{}_mask.bed --recode --keep-INFO-all --stdout | bgzip -i -I filtered/{}.tbi > denisovan.filtered.{}.vcf.bgz '

Vcftools is running on 4 files, but it is still extremely slow. Each vcf.gz file is 3-4gb in size, but having had vcftools running for almost 24 hours now, the four files still have not been completed. Is this normal? I am aware that I will be limited by processor speed but as this will take several days before it is completed I wanted to check that there is no way to optimise this process. Thanks.

• 0 views • link
written by spiral01 spiral01

Log in to answer this question.

More posts like this