This is a test version of Biostars. For the public version, visit https://www.biostars.org.
BWA script on partially executes

I'm trying to run multiple alignments on my school's pbs cluster. Running the script as an interactive job works perfectly fine, I get all the files .sam files I need in about an hour. When I run it as a non-interactive job it outputs about a 9000 kb .sam file and then finishes the job which is far from complete. 

#!/bin/bash
#PBS -l nodes=1:ppn=28,walltime=02:00:00
#PBS -N mapping
#PBS -q short

cd /home/workingdir/

module load shared
module load anaconda/2

i=1

for j in {1..10};
  do
    bwa bwasw -t 3 "mypath/ref.fasta"  'CH2009_split_'$j'_R'$i'.fq.gz' > map/'CH2009_'$j'_R'$i'.sam' &
done

This is the same script I run in the interactive and non-interactive node. I should also say that I've split up one large .fasta file into smaller ones so I can align them simultaneously in the for loop

alignment bash pbs
• 1,211 views • link
updated by Philipp Bayer Philipp Bayer • written by matthew.dambrosio.1 • matthew.dambrosio.1

1 answer

Some PBS systems don't like the & - the jobs get all sent to background, the first job starts writing, but since all jobs are in the background the PBS job finishes.

Have you tried adding a single wait at the end of the last line? Like in this example: https://stackoverflow.com/a/13283453

And since you have so many jobs, it's probably better to learn about PBS job arrays (one job per 'slot' in the array) https://arc-ts.umich.edu/software/torque/job-arrays/

• 0 views • link
written by Philipp Bayer Philipp Bayer

Log in to answer this question.

More posts like this