Hello,
I am trying to perform a blastp search of 23 sequences against a BLAST database containing approximately 15,000 sequences (in a for loop that will give me seperate outputs). As a newcomer to HPC systems, I am unsure how many CPUs or how much memory I will need for this task. Can you advise me on how to assign these parameters correctly? Is there a way to determine the optimal values or will my intuition improve with experience?
1 answer
RAM needs depend on lengths of sequences as well.
Your database is small (you can see the file size). So just over-provision - based on the standard node size in your cluster.
Normally you should avoid doing a for loop... because that will require more I/O and Cpu than doing all as a batch. (but here your numbers are quite small, so the cost of extra cpu cycles will be negligible)
The way BLAST is engineered, people typically run one multithreaded ("batch") BLAST analysis (i.e., with one multi-Fasta format query file). You probably want to benchmark some of your analyses to see how the RAM/CPU needs scale.
Log in to answer this question.
-
Optimizing GROMACS MD performance and reducing queue time on HPC (Slurm cluster)
written by DAUNTLESS •
-
blastp for short sequences not finding a sequence that I know is in the database
written by ricfoz •
-
alphafold hpc large fasta file job
written by Kayenat Sheikh •
-
Designing single-stable RNA molecules
written by Edna •
-
How do I interpret BLAST results
written by Alex •
-
BLAST Reciprocal Best Hits (Linux)
written by tannerndrsn4 •
-
Trace element utilisation project
written by reddishvilla_12 •
-
Trace element utilisation project
written by reddishvilla_12 •
-
Bulk Segregant Analysis using next generation sequencing data
written by p.weru •
-
Phylogenetic Analysis Of Protein Expression
written by Griffin Lester •
If your HPC uses a job scheduler then submit the jobs via that mode. You don't say what is the size of the database and query but assuming they are not like
nt/nryou may be able to get away with 8 cores and 30G of RAM. You may simply need to run a few jobs and try things out.I don't know if there is a job scheduler, I'll look at it. I don't exactly remember but my database was ~30 megabytes. Since it's a small one with 4 cores and 32 GB of ram handled my job in less than 2 min. I gave query sequences one by one with a for loop and get separate outputs. I didn't have the chance to try this with databases of different sizes, but I'll try different scenarios and will learn from them. Thank you for your answer!