Mutect2 is not detecting all available cores (multithreading)

Question

[jgarces]

Hi,

I'm trying to run _Mutec2_ with multiple threads but something's going wrong. This is my code:

gatk Mutect2 --java-options "-XX:ParallelGCThreads=32" \
    --native-pair-hmm-threads 32 \
    -R hg38.fa --intervals hg38.intervals \
    -I tumor.bam -I normal.bam -tumor tumor -normal normal \
    -O tumor.mutect.vcf

Everything goes well but it's only detecting 1 core...

09:32:24.816 INFO  IntelPairHmm - Using CPU-supported AVX-512 instructions
09:32:24.816 INFO  IntelPairHmm - Flush-to-zero (FTZ) is enabled when running PairHMM
09:32:24.817 INFO  IntelPairHmm - Available threads: 1 <-----
09:32:24.817 INFO  IntelPairHmm - Requested threads: 32 <-----
09:32:24.817 WARN  IntelPairHmm - Using 1 available threads, but 32 were requested <-----
09:32:24.818 INFO  PairHMM - Using the OpenMP multi-threaded AVX-accelerated native PairHMM implementation
09:32:24.856 INFO  ProgressMeter - Starting traversal
09:32:24.856 INFO  ProgressMeter -        Current Locus  Elapsed Minutes     Regions Processed   Regions/Minute

Any idea what I'm missing, please?

[GenoMax]

Everything goes well but it's only detecting 1 core...

It is using one thread but does say that you asked for 32. You do have 32 threads available, when needed?

Any idea what I'm missing

Probably nothing. Programs may not use all cores at all steps. Keep checking the logs.

Answer 1

[jgarces]

I think I fixed this problem (on SLURM). This comes from how GATK is configured for parallelizing jobs: it's based on OpenMP, so it's needed to add in the beginning of the script something like this:

export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK

(source)

[chatul]

Thank you for returning to add the solution! This just solved my problem as well. In case useful for others, adding the solution for PBS users - add ompthreads=W, with W being the number of OpenMP threads in the qsub command or the header eg

#PBS -l select=N:ncpus=Y:mem=Z:ompthreads=W

Answer 2

[mad_nerd]

Hi, It looks that you have only 1 thread available on your machine.

09:32:24.817 INFO  IntelPairHmm - Available threads: 1 <-----

I had similar problems with GATK but it was due to my computational set-up. Maybe you will find a bug or any information about the version/release that you are using

https://github.com/broadinstitute/gatk/releases

[jgarces]

I'm using an HPC environment and requesting 32 cores, so I'd expect to have this amount of cores... but something happens and these cores never appear.

[GenoMax]

This may be difficult to diagnose on a forum but what job scheduler are you using and are you asking for the same allocation (32 cores) in that part of the command line/script.