This is a test version of Biostars. For the public version, visit https://www.biostars.org.
CD-HIT uses up all RAM and then crashes

I'm trying to use cd-hit to generate a 0.9 sequence identity cutoff of the Blast NR database.

Here's what I'm running:

cd-hit -i nr -o nr90 -c 0.9 -M 1000

But, even though I'm using the -M 1000 option, the command just gradually uses up all the available RAM (8gb) and then crashes. Any idea on how to fix this?

cd-hit
• 2,522 views • link
updated by Biostar Biostar • written by VDL • VDL

More RAM.

if you want to cluster all of nr, you’re going to need much more than 8gb.

This strikes me as an XY problem though, what are you trying to achieve?

• 0 views • link
written by Joe Joe

I'm trying to replicate a result for a protein prediction problem that used this database. My understanding was that the -M flag was supposed to limit the amount of RAM that the program used. So it doesn't work?

• 0 views • link
written by VDL • VDL

The program needs to use at least a certain amount of RAM. You can’t make a program that needs X gb of RAM run on < X.

It’s probably hitting your limit of 1gb, and then crashing, RAM usage can be a somewhat complex thing to monitor. I’m not sure why it continues to use all 8gb when you specifiy 1, but regardless, NR is far, far too big to be done with even 8, I’d pretty much guarantee.

• 0 views • link
written by Joe Joe

Maybe you want to use UniRef?

• 0 views • link
written by h.mon h.mon

I'll probably use it if I can't get the other option to work, thanks for pointing that out.

• 0 views • link
written by VDL • VDL

0 answers

No answers yet.

Log in to answer this question.

More posts like this