This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Split chromosome into smaller fragments to save memory when build xg and gcsa index.

I follow the tutorial to build index for each chromosome. This saves memory when build index. However, my pc run out of memory when building xg index even in this case. Is it possible to split chromosome into smaller fragments to further save memory when build xg and gcsa index? How to specify this in the -R option of vg construct?

vg
• 5,169 views • link
updated by Jordan M Eizenga • Jordan M Eizenga • written by 李经纬 • 李经纬

I have tried to index various chromosome sets.

  • chrom1 only: Memory usage: 2.53022 GB.
  • chrom1-10: Memory usage: 13.3299 GB.
  • chrom1-20: Memory usage: 22.9164 GB.
  • chrom1-35: Memory usage: 23.5126 GB.

The memory usage will increase when more chromosome is included.

• 0 views • link
written by 李经纬 • 李经纬

I use vg construct --region chrom:start-end to restrict graph to a chromosome region.

• 0 views • link
written by 李经纬 • 李经纬

1 answer

I would say the answer is unfortunately no. It would be possible to make smaller individual XG indexes, but downstream subcommands in the VG toolkit do not support multiple XGs. In general, VG is designed mostly for HPC clusters and shared compute servers that have > 64 GB of RAM. It would be difficult to run VG on most consumer-grade PCs.

• 0 views • link
written by Jordan M Eizenga • Jordan M Eizenga

Thank you. Even if I build the XG index after some modifications, the building of gcsa failed even on a single chromosome. Even though I have pruned the vg graph before building gcsa.

• 0 views • link
written by 李经纬 • 李经纬

GCSA2 indexing is fairly demanding on disk space, so that can happen. I would generally recommend having 1-2 TB of disk availability if you're looking at a eukaryotic genome. It may be possible to increase the stringency of the parameters in vg prune in order to use less memory, but it's difficult to predict what the best pruning parameters would be.

• 0 views • link
written by Jordan M Eizenga • Jordan M Eizenga

Log in to answer this question.

More posts like this