There was some local interest in running a parallelized version of the meme
binary in the MEME suite (http://meme-suite.org/). Many moons ago, I had to do this for an environment that ran under a Sun Grid Engine and posted a question here about it.
MEME has undergone some revisions and we've changed job schedulers, away from SGE and to Slurm. Recently, I put together a makefile
that demonstrates how to build an OpenMPI-capable meme
binary (one that can run meme -p <num_processors>
) and includes some targets that test this binary locally and on a cluster that uses a Slurm job scheduler.
This assumes that the end user has a cluster that is managed with a Slurm job scheduler, and that the Slurm nodes use the modules
system to manage software packages.
Starting with a {data,results,src}
-structured project folder, one could put the following makefile
in the src
folder and run targets.
One can make the download
, extract
, and build
targets to compile MEME 4.12.0 with OpenMPI support.
For my setup, it is necessary to hardcode paths to Ghostscript gs
and ImageMagick convert
, as the module system puts these binaries somewhere other than /usr/bin
, which is where meme
would otherwise look. Depending on your module setup, this may not be required.
Changing the cluster partition on which jobs will run will also almost certainly be a necessity.
Review the test_local
and test_slurm
targets to see how you can run meme -p
"locally" (on a host that also has modules support and multiple cores/processors) and on a Slurm-based cluster.
The Slurm-focused test_slurm
target creates ../results/slurm_32
and ../results/slurm_64
directories. The stderr.txt
file in each directory returns their runtimes.
It may be useful to test adjustment of Slurm nodes
, ntasks-per-node
and cpus
parameters to match the -p <N>
argument and review overall performance. This will be specific to the local environment.
Hope this is useful to others!