how to speed up blastx run on slurm HPC

Hi,

I am using below script for balstx against the ncbi nr database but it is taking so much time, and I got around 150000 sequences in fasta file? Is there any way to speed up the analysis?

#!/bin/bash
#SBATCH --job-name=nr      # Job name
#SBATCH --mail-type=END,FAIL         # Mail events (NONE, BEGIN, END, FAIL, ALL) 
#SBATCH --ntasks=40                  # Number of MPI ranks
#SBATCH --nodes=2                    # Number of nodes
#SBATCH --time=8000:00:00              # Time limit hrs:min:sec
#SBATCH --partition=k2-lowpri
#SBATCH --mem=200G

module load apps/ncbiblast/2.10.0/gcc-7.2.0

blastx -query sequecnes.fasta -db /mnt/scratch/ncbi/nr_protein/nr -evalue 1e-5 -outfmt "7 std stitle qseqid sseqid pident length mismatch gapopen qstart qend qcovs qlen slen" -max_target_seqs 1 -out nr.new.txt

Many thanks

bash unix hpc

• 3,030 views • link

updated 5.9 years ago by Mensur Dlakic

• written 5.9 years ago by Bioinfonext

2 answers

By using a corresponding request in your blastx command line of -num_threads to match --ntasks. I would keep the threads on one node so change --nodes to 1.

• 0 views • link

written 5.9 years ago by GenoMax

thanks genomax,

so should I change the resources like this:

#!/bin/bash
#SBATCH --job-name=nr      # Job name
#SBATCH --mail-type=END,FAIL         # Mail events (NONE, BEGIN, END, FAIL, ALL) 
#SBATCH --ntasks=40                  # Number of MPI ranks
#SBATCH --nodes=1                    # Number of nodes
#SBATCH --time=8000:00:00              # Time limit hrs:min:sec
#SBATCH --partition=k2-lowpri
#SBATCH --mem=200G

many thanks

• 0 views • link

written 5.9 years ago by Bioinfonext

Correct. As long as individual nodes have 40 cores. If not drop that number to match what your nodes have. You can also used DIAMOND (LINK) as a faster alternative. You will need to create your own indexes though.

• 0 views • link

written 5.9 years ago by GenoMax

Add -num_threads 40 to your blastx command.

• 0 views • link

written 5.9 years ago by Mensur Dlakic

thank you so much! now I understand it.

• 0 views • link

written 5.9 years ago by Bioinfonext