This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Tool: vcfkit — open-source Rust CLI for VCF normalize, liftover, and filter (4× faster than bcftools on filter)

I built vcfkit, an open-source command-line toolkit for three VCF operations every pipeline needs: normalize, liftover, and filter.

Why I built it: I wanted a single static binary with no htslib dependency that could also run in the browser via WebAssembly. bcftools is the gold standard and vcfkit validates against it — if they diverge, vcfkit is wrong by default.

Performance on 1000 Genomes chr22 (1.1M variants):

Operation vcfkit bcftools Speedup
filter -e 'INFO/AF < 0.01' 422 ms 1,695 ms 4.0×
normalize --fast --no-split 682 ms 2,820 ms 4.1×
normalize (standard) 6,481 ms 2,820 ms 0.43× slower

The fast paths use raw-line parsing, only parsing fields the expression touches. Standard normalize is slower because it uses a pure-Rust parser (noodles) rather than htslib. I documented both numbers honestly rather than hiding the slow path.

Install: cargo install vcfkit-cli

Pre-built binaries for macOS, Linux, Windows at the releases page — no Rust required.

Browser demo: vcfkit.dev — all three operations run in WebAssembly client-side. Your VCF never uploads anywhere.

Natural-language filter (experimental): vcfkit filter --ask "rare variants in European populations" input.vcf

Translates plain English to a filter expression via Claude, shows you what it will run, and asks before running. Requires your own ANTHROPIC_API_KEY. The LLM sees only the VCF header schema, not variant data.

Correctness: validated against bcftools via differential tests on real 1000G chr22 data. Tests run nightly in CI. Known behavioral differences from bcftools are documented.

A note on my background: I'm an indie developer, not a bioinformatician. I learned the domain while building this and validated everything against bcftools because I knew I couldn't trust my own domain intuition.

Credits: built on the work of htslib/bcftools (Heng Li, Petr Danecek, Wellcome Sanger Institute), noodles by Michael Macias, and the Tan/Abecasis/Kang 2015 normalization algorithm. Portions of the code were written with AI assistance (Claude) and verified by a human against bcftools output. See CREDITS.md for full attribution.

Links:

What would help most: if you work with VCFs regularly, running vcfkit on a VCF you actually use and reporting what breaks or what's missing. Correctness bugs, unsupported expressions, performance issues on files larger than 1M variants — all useful. Open a GitHub issue, reply here, or email me at prasadkhake@gmail.com.

Happy to answer questions.

rust vcf normalize filter liftover

0 answers

No answers yet.

Log in to answer this question.