Hi all, I am trying to optimize an RNA-seq pipeline and I want to be able to estimate the RAM requirements for fastq files of different sizes. So far I have tested on files from typical rna-seq expeirments of ~30 to 40 million reads. I want to now test on much larger data where the file is close to 50gb in size .
I was wondering where I can obtain such files for testing. Can anyone point me to some publicly available datasets that have more number of sequences than what I have already done? Anything like >= 150 million reads would also be okay.
Thanks , RK
•
written
by
ramyak1912 •
1 answer
ENCODE has some samples with more than 100m reads in them. E.g. ENCSR000COU. When you talk about fastq size are you talking about compressed or uncompressed? Because compressed, even this 100m read sample is only ~ 8Gb.
BTW, with most of the standard pipelines (STAR -> featureCounts -> DEseq2 or salmon -> tximport -> DESeq2 or kalisto -> tximport -> DESeq2) memory usage scales with the size of the genome, and the number of reads doesn't make a difference.
Log in to answer this question.
-
Minimum reads per sample
written by rico07 •
-
normalize different size fastq files
written by serene.s •
-
RNA-seq dataset search
written by ramyak1912 •
-
Hardware requirements for bowtie2/STAR RNA-seq alignment
written by MatStat
-
Genome Size Estimation
written by Mbillah
-
merging fastq, sam or bam?
written by ceboral •
-
BBmerge histogram of insert size
written by Sharon
-
RNA-seq quality control
written by Ron
-
rRNA detection (for contamination) in RNA-seq
written by Ron
-
Selection between Illumina Hiseq 2500 and NextSeq
written by ebrahimiet •
You can try to merge multiple samples to big fastq file. Use
catorzcatcommand.Hi , Thanks for your reply. I used some samples from ENCODE which were ~30GB or ~200 million reads to run some jobs on aws batch. And I got Out OfMemory Errors. And I am aligning to the human genome. I used the same pipeline before for running a basic rna-seq experiment with 25 to 30 million reads and I didn't have any problems then.
Do you have any idea about what could be the problem?
WIthout details on the pipeline this is impossible to answer. Please add comments via
ADD COMMENT/REPLYto keep things organized.