This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Speed of merging multiple bcfs with bcftools compared to PLINK?

I am trying to merge 3 sets of bcfs that contain hundreds of millions of sites for tens of thousands of samples. These bcfs have the exact same sites, just all different samples. using bcftools merge 1.bcf 2.bcf 3.bcf -Ob > merged.bcf looks like it is going to take days, maybe even weeks at the rate it is going. Even though they are both binary formats, would it be faster to first convert these each to plink then:

plink --make-bed --merge-list merge_list.txt --out merged

where merge_list.txt is a list of my binary plinks for each bcf:

1
2
3
bcftools plink

1 answer

bcftools merge supports multiple threads. Looks like you are using just one.

--threads <int>                use multithreading with <int> worker threads [0]

Long as your storage subsystem is up to the task use multiple threads to speed things up.

Same with plink.

Oh darn it. thank you!

Log in to answer this question.