This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Maximum size limit for indexing genome with bowtie2 with large index option

Hello all

I have a fasta file with 100 GB of data and I am trying to create a bowtie2 index. It is showing the following error:

Ran out of memory; automatically trying more memory-economical
parameters. Using parameters --bmax 54 --dcv 4096

Any help would be appreciated. Thanks in advance

genome alignment assembly software error

How much memory does the machine have? 100GB is quite large, so you'll need a reasonable amount of memory to even create the index.

I am running on a 256 gb RAM machine with 35 cores

So it's quite possible that there is not enough free memory to build the index with Bowtie2's default settings. You have only ~2.5x as much memory as the size of the input file, and this is likely a shared machine where other processes may be using memory. You can attempt to reduce Bowtie2's indexing size using parameters like --packed and --offrate 7 or --offrate 8 (these latter ones will make mapping slower). Also, it's worth noting that it looks like the message you posted isn't an error. It's a message that lets you know the default parameters are inadequate, and the wrapper is trying to change the parameters to make the index building succeed on your machine. --bmax and --dcv are other parameters that affect the memory required for indexing, and so it's setting them to more conservative values.

Thank you. Will look into it.

Is this 100Gb a single genome? What are you trying to do?

It is a multiple genome fasta file

...and why would you align to such a collection?

0 answers

No answers yet.

Log in to answer this question.