Tool: vcfv - fast VCF parser to Polars/Pandas DataFrame with native INFO and FORMAT expansion
Hi, some time ago I created a tool for parsing VCF files with native INFO and FORMAT expansion support.
After a casual conversation with some friends, I came to the conclusion that it might be worth sharing it.
pip install vcfv
Benchmarks on 500k variants:
| Mode | Time | RAM |
|---|---|---|
| vcfv (INFO+FORMAT expanded) | 3.39s | 570 MB |
| vcfv (no expansion) | 0.15s | 273 MB |
| scikit-allel | 2.10s | 115 MB |
| pandas raw TSV | 3.23s | 358 MB |
vcfv (no expansion) vs scikit-allel - fair apples-to-apples: ~14x faster.
vcfv is the only tool here that natively expands INFO and FORMAT fields.
Quick example:
df = read_vcf(
"example.vcf",
parse_info="wide",
parse_format="wide",
)
Known limitation:
.vcf.gz files are fully decompressed into memory before parsing.
Native bgzf/tabix streaming is planned for a future release.
GitHub:
https://github.com/rvk20/vcfv
PyPI:
https://pypi.org/project/vcfv/
I'd love to hear your feedback!
0 answers
No answers yet.
Log in to answer this question.
How does it compare to polars-bio scan_vcf method?
I actually wasn't aware of polars-bio before, so I ran a quick comparison.
The main difference I noticed is in approach: vcfv flattens INFO and FORMAT fields into flat, wide columns by default (Sample1_GT, Sample1_DP, etc.), which I find convenient for downstream analysis without unpacking nested structures. In RAM usage, polars-bio clearly has the edge.
I also noticed polars-bio supports lazy .vcf.gz streaming, S3/GCS loading, coordinate system validation, and genomic interval operations, features that are out of scope for vcfv.
vcfv is a lightweight tool designed for working with small and medium-sized VCF files where simplicity matters more than performance at scale. polars-bio is the better choice for production genomics pipelines.