This is a test version of Biostars. For the public version, visit https://www.biostars.org.
rsem-calculate-expression ran out of memory

I first built the terence with the following command

rsem-prepare-reference --bowtie reference_hg19.fa hg19_ref

And then ran the following:

rsem-calculate-expression --paired-end PairedEnd1.fastq PairedEnd2.fastq hg19_ref my_test_sample

After running for about 20 hours, all I got was the following output:

---------------------------------------------------- start ----------------------------------------------------
bowtie -q --phred33-quals -n 2 -e 99999999 -l 25 -I 1 -X 1000 -p 1 -a -m 200 -S hg19_ref -1 PairedEnd1.fastq -2 PairedEnd2.fastq | samtools view -S -b -o my_test_sample.temp/my_test_sample.bam -
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:1:12107:15053:9521 1:N:0:CAGAGAGG/1 (
patid 4851744); skipping read
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:1:21109:16613:17366 1:N:0:CAGAGAGG/1 
(patid 13368267); skipping read
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:2:13106:13072:15868 1:N:0:CAGAGAGG/1 
(patid 33771310); skipping read
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:2:21111:9396:9031 1:N:0:CAGAGAGG/1 (p
atid 38469910); skipping read
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:2:23211:19862:18298 1:N:0:CAGAGAGG/1 
(patid 48288092); skipping read
---------------------------------------------------- end ----------------------------------------------------

So I cancelled it.

It was ran on a server which had over 300GB memory, so I don't think it's the problem of the physical memory we have. Anyone has any idea?

Thanks a lot in advance!

-- m.x.

rna-seq
• 5,807 views • link
updated by davedeto • davedeto • written by moxu moxu

I reran rem-calculate-expression with added parameter "-p 8", but go the same warning for the "chunk memory".

• 1 views • link
written by moxu moxu

-p option is for cores to use. Option you want to change would be --bowtie-chunkmbs <int> Default is 64 MB so set it to something bigger.

• 1 views • link
written by GenoMax GenoMax

What <int> would you suggest? Some people use 256, but why not set it to even bigger, say, 1024?

Thanks!

• 1 views • link
written by moxu moxu

Very helpful, and I joined the RSEM users google group. Thank you!

• 1 views • link
written by moxu moxu

1 answer

Not sure if you found your answer yet, but the issue is probably that you are using the whole genome, and not just the transcriptome (expressed transcripts only).

Read the examples here (http://deweylab.biostat.wisc.edu/rsem/rsem-prepare-reference.html) when you prepare your reference. Either the reference file already needs to be split into transcripts (if you open it, you'll see one fasta header, ">", for each transcript, not each chromosome), or else you need to call it with the -gtf option, and supply a GTF file that tells RSEM where the genes are.

• 1 views • link
written by davedeto • davedeto

--gtf & -p made the processing work. Thanks a lot!

• 1 views • link
written by moxu moxu

Log in to answer this question.

More posts like this