Has anyone compiled and used MEME 4.x for use in a parallel computation environment, based upon operation with a Sun Grid Engine (SGE) cluster?
I can compile the suite and its tests pass. However, when I attempt to use the -p n option, to specify n computation nodes, I get several error messages:
/gridware/codine/util/arch: Command not found.
/gridware/codine/util/arch: Command not found.
/gridware/codine/util/arch: Command not found.
/gridware/codine/util/arch: Command not found.
1: Command not found.
We do not have /gridware/codine/util/arch, but we do have /gridengine/sgi/util/arch.
I tried looking around MEME's source code, particularly at meme.c and mp.h, but there are no references to these paths.
I'm wondering if I am missing makefile directives. Here is my ./configure statement:
./configure --prefix=/home/areynolds/proj/meme/meme_4.3.0_build --with-url="http://meme.nbcr.net/meme" --enable-openmp --enable-debug
Is MPI a requirement; are there directives I am missing for MPI?
Thank you for any advice.
EDIT
I was able to successfully build a version of MEME 4.3 that supports OpenMPI.
First, I worked with our sys admin to install OpenMPI 1.4 on each of the SGE nodes (compilation options) and set up an SGE parallel environment called mpi_test:
$ qconf -sp mpi_test
pe_name mpi_test
slots 120
user_lists NONE
xuser_lists NONE
start_proc_args /bin/true
stop_proc_args /bin/true
allocation_rule $fill_up
control_slaves TRUE
job_is_first_task FALSE
urgency_slots min
Secondly, I used the following build options for MEME:
$ ./configure --prefix=/home/areynolds/proj/meme/meme_4.3.0_build \
--with-url="http://meme.nbcr.net/meme" \
--enable-openmp \
--enable-debug \
--with-mpicc=/opt/openmpi-1.4/bin/mpicc \
--enable-opt
Thirdly, the mpirun and meme_p binaries depend upon shared libraries in the OpenMPI installation. It is necessary to add /opt/openmpi-1.4/lib to your local LD_LIBRARY_PATH environment variable before running qsub.
Finally, I set up a script (called runall.cluster) along the following lines. Runtime options can be adjusted to taste:
#!/bin/bash
#
# runall.cluster
#
#$ -N memeCluster64
#$ -S /bin/bash
#$ -pe mpi_test 64
#$ -v -np=64
#$ -cwd
#$ -o "memeCluster64.out"
#$ -e "memeCluster64.err"
#$ -notify
#$ -V
time /opt/openmpi-1.4/bin/mpirun -np 64 \
/home/areynolds/proj/meme/meme/bin/meme_p \
/home/areynolds/proj/meme/data/K562.DS9767.fps.not.in.promoters.fa.top10k.fa \
-oc /home/areynolds/proj/meme/output/K562.DS9767.fps.not.in.promoters.10k.cluster.64 \
-maxsize 230000 \
-dna \
-minw 8 -maxw 12 -allw \
-p 64
This script is executed directly with qsub and runs the parallelized MEME binary meme_p with the specified options. In this case, it will run on 64 nodes of our computation cluster:
qsub ./runall.cluster
Hopefully this information will help out others.
2 answers
I've put a link on the MEME User Forum to this thread.
This may not be overly useful but it very much sounds like a configuration problem.
Usually there is configure flag that needs to be set to point to the libraries, something like:
--with-mpidir=MPIDIR
--with-mpicc=MPICC
It also appears that the MEME suite does not support Open MPI (as per install notes).
I would also recommend posting on the MEME user forum
Log in to answer this question.
thanks for following up on this, great information
This is the real spirit of Biostar, thanks for sharing this.