BWA batch script does not execute

Question

[madkitty]

I'm running an alignment with BWA on multiple exomes files on our server. For some reason it does not output anything, and doesn't even seem to be working... any clue why ?!?!

#!/bin/bash
#PBS -l nodes=2:ppn=8
#PBS -l walltime=5:00:00
#PBS -N my_username
cd $PBS_O_WORKDIR

module load bwakit

for i in $(ls *.fastq)
do

bwa mem /mypath/ref/hg38.fa -t 8 /mypath/R1/$i_R1.fastq /mypath/R2/$i_R2.fastq
> /mypath/align/$i\.sam
${i%.fastq}

Done

[Devon Ryan]

Try executing it on the login node and see what happens...

Hint: done, not Done

[GenoMax]

Noooo ... are you trying to get @madkitty banned from this cluster :-)

[Devon Ryan]

Eh, a small pair of input fastq files and lowering the thread count would have made it apparent what the problem was. At least for the clusters I've used there's been a bit of understanding when people try to debug something small on the login node.

[GenoMax]

Per the other thread I linked below @madkitty has 100 files (50 jobs).
I am not a PBS user but at least on LSF every job that is submitted needs to be encapsulated with LSF commands. That is not going to happen with the script above. Perhaps PBS works differently.

[Devon Ryan]

My expectation is that this is just a scripting error, which would then be apparent running things on a single server.

[GenoMax]

Re-reading the OP it is not clear if the script is running or it is running and the actual jobs (not sure if the script as written will produce 50 independent jobs) are not producing any output. Part of the problem is likely the file name issue that @Brice has addressed in his answer.

[madkitty]

The script is just not doing anything, no files are being creating.

[GenoMax]

@madkitty: We had gone over how you would do this in the other thread (C: How to align 100 samples with BWA? ). You are not following the advice there.
You would do module load bwakit and then follow the script in thread mentioned.

[madkitty]

Yes I followed the advices there.

Answer 1

[Brice Sarver]

Four things, some minor:

1. Make sure the call in bwakit module to execute BWA is actually bwa. Perhaps try on the head node, but don't run it unless you want to get yelled at by your Sys Ad.

2. Variable 'i' is equal to '[STRING].fastq' Your call, therefore, is malformed. You want to do something like:

   name=$(echo $i | cut -d '.' -f 1); and then let 'name' be the first part of the fastq.

   Edit: I see that you were trying to do this with the % operator, but you need to define this beforehand: name=$(echo $i | cut -d '.' -f 1); or b=${i%.fastq} would store the same value, assuming that it follows the simple structure listed above.

3. for i in *.fastq; do ... is all you need in this case.

4. You are using 8 cores, but you are reserving two nodes with 8 processors per node. Seems like nodes=1:ppn=8 would suffice.

5. [Devon already commented on the 'Done' issue above.]

[madkitty]

Thank you, will give it a try with your advices.