Do People Import VCF Files Into Databases? (2016 version)

Question

[leekaiinthesky]

This is a duplicate of:

1. Do People Import Vcf Files Into Databases? from 5.4 years ago and
2. Which Type Of Database Systems Are More Appropriate For Storing Information Extracted From Vcf Files from 3.5 years ago.

Nevertheless, I found the answers on the second post enlightening, and 3.5 years later it seems worthwhile to have an update from the community (and @lh3) on how you deal with these issues.

Notably, @Aaronquinlan's gemini from the second most upvoted answer to question #2 is still under active development. Are people using it? What else are people doing?

[Eric Lim]

If I need to query the contents in the VCF, especially large ones like ExAC, I normally use HDF5.

Answer 1

[Jeremy Leipzig]

This is called "variant warehousing" and there are several open source and commercial efforts in various stages:

Golden Helix VSWarehouse
Paradigm4 SciDB
WuXi NextCODE
CMH Variant Warehouse
ViaGenetics Genesis
Curoverse Lightning
Intel GenomicsDB
Cloudera OMICS

Answer 2

[rbagnall]

PlinkSeq can read vcf files into a database format, then generate summary stats and extract gene, individual or cohort level data. There is an online tutorial that is helpful.