In a nutshell, I have 44 folders of different samples/species that each have paired reads for those samples/species. I'm doing bowtie alignment with the same referent genome, and then outputing it to BAM and sorting it using samtools. Since alignment takes a while, I've written a script and passed it to Sun Grid Engine job manager with parallel job options. The code for that is:
#!/bin/bash
#$ -N bowtiejob
#$ -V
#$ -t 1-44 #this creates task IDs, numeric indexes
read -a samples <<< $(cut -d , -f 3 linker.csv | tail -n +2) #list of all sample folders
false_index=$SGE_TASK_ID #this is just for ease of writing
true_index=$((false_index-1))
folder=${samples[$true_index]}
ref_path="ref_genome/ref_genome_btindex" #this is my Bowtie index ref genome
With the above code, I ensure that every sample folder is assigned unique SGE_ID so that parallel jobs don't interact with one another.
The code for bowtie and samtools is following:
bowtie2 -x $ref_path -1 $folder/*_1.fastq -2 $folder/*_2.fastq | samtools view -bS - > "bowtie/$folder.sat_ref.bam"
samtools sort bowtie/$folder.sat_ref.bam -o bowtie/$folder.sorted.bam2
The paired reads are found in separate folders, and referent genome index built with bowtie2-build is specified in $ref_path. The first line of code passes bowtie result to SAM, then SAM is converted in BAM. The second line of code creates a sorted BAM file from BAM file.
The script is literally the same, it's executed the same, with just a difference in $SGE_TASK_ID. However, some samples run as expected, while some don't. For example
sample01.sat_ref.bam
sample02.sat_ref.bam
The first sample has 0 KB, while the second one has 13M Kb. Seeing the error output and output for failed sample - the output is empty and the error output is :
/usr/local/bin/bowtie2-align-s: error while loading shared libraries: libtbb.so.2: cannot open shared object file: No such file or directory
(ERR): Description of arguments failed!
Exiting now ...
samtools: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by samtools)
samtools: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by samtools)
Likewise, the error output for the good sample is empty, while output for it is :
64055526 reads; of these:
64055526 (100.00%) were paired; of these:
28974485 (45.23%) aligned concordantly 0 times
27752984 (43.33%) aligned concordantly exactly 1 time
7328057 (11.44%) aligned concordantly >1 times
....etc
My guess is that job failed because of whatever is causing this error
/usr/local/bin/bowtie2-align-s: error while loading shared libraries: libtbb.so.2: cannot open shared object file: No such file or directory
(ERR): Description of arguments failed!
But I have no idea why it failed for some samples, and succeeded for other.
