I am running cuffdiff on about 20 samples, and I simply started a single job for the whole genome. It is still running (and still producing output, so it isn't "stuck" or anything) 5 days later. I think I could parallelize it by splitting my input GTF file into many small sets of genes, but I am worried that it would produce incorrect FDR calculations and other "aggregate" statistics that I would not easily be able to correct when I merge the output.
Is there an easy way to parallelize cuffidff, or must I wait for my single job to finish?
•
written
by
Ryan Thompson
1 answer
Be sure to use the "-p" option, as Aaron suggests in his comment. And, yes, splitting into small sets of genes is not the way to go for the reasons that you suggest.
Log in to answer this question.
-
TEdenovo blasting with Recon perpetuates..!
written by mthm •
-
fgsea: What does fgseaMultilevel argument sampleSize mean/when to change it?
written by kelen
-
Blast job never stops
written by langziv •
-
Stuck at equilibration step6.1
written by asma.nounna •
-
Fail job after all vs all : Error, 'unable to ReadProgram on SLURM cluster
written by gusa_10 •
-
popoolation parallelization problem
written by dovi
-
topGO which statistical test (fisher or KS) to use ?
written by mary •
-
Cuffdiff hangs at "Inspecting maps and determining fragment length distributions" step
written by tunl
-
Blastx With Xml Format
written by jinzhangut2010 •
-
Getting Watson Snps From Biomart
written by Andrea_Bio •
Are you using the "-p" option to use multiple threads?
Hmm. I am running it on a cluster, and I have not experimented with multi-threading on the cluster yet (instead of one job with N threads, I run N jobs with one thread each). I know my cluster allows multi-threaded jobs, but only up to the number of CPU cores available on a single node. Still, it's better than nothing.
In the end, where you able to get it to run successfully? If so, how long did it take (and how much memory/processors)? I'm currently running 45 individuals on 24 cores--which hasen't been that useful because after the mapping stage cuffdiff seems to revert back to 1 core.
In the end, were you able to run it successfully? If so, how long did it take (and how much memory/processors)? I'm currently running 45 individuals on 24 cores--which hasen't been that useful because after the mapping stage cuffdiff seems to revert back to 1 core. Any thoughts appreciated!
No, my internship ended. I'm now using Cufflinks again on a completely unrelated project, this time on a single 8-core workstation. We'll see how things go this time.