Hi all,
I want to make a parallel shell script, which could run the trimming processes parallel on different processors and later do the genome assembly. I am trying the following script:
Before doing this, one could also split the whole read file into small chunks (using split command) and use for loop to do the trimming on multiple processors.
#!/usr/bin/bash
STEP1: Library1 read Filtering/Trimming & # Use any software of your choice
STEP2: Library2 read Filtering/Trimming &
STEP3: Library3 read Filtering/Trimming &
wait
velveth demo 65 -short -fastq Library1 -short2 -fastq Library2 -short3 -fastq Library3
velvetg demo -exp_cov auto
Dose anybody have better way to write this? Which would also take care of successful completion of first 3 steps. using their process Ids.
I would really appreciate your inputs. Thanks in advance!
Best regards,
Rahul
•
written
by
Rahul Sharma
2 answers
Your 2 inquiries are quite different:
1 - the trimming step is very easily parallelizable as you don't need the information in the other sequences to process one, I think the best way to do it in bash would be by using GNU parallel (take a look here: GNU Parallel - parallelize serial command line programs without changing them ) . However, I've been processing quite a lot of big datasets but I've always figured it was fast enough using a single processor and I'm wondering if this worths doing it in parallel.
2 - the assembly step is much more complex and I think you cannot parallelize an assembler which has not been designed for. Until recently I did not know about any assembler capable to do it, but now there is Ray and maybe (but not sure it'd fit your needs) Minia, but I personally don't have any experience with any of those.
use a makefile with option -j (num parallel jobs)
VELVET2: VELVET1
velvetg demo -exp_cov auto
VELVET1: Library1 Library2 Library3
velveth demo 65 -short -fastq Library1 -short2 -fastq Library2 -short3 -fastq Library3
Library1:
statements1
Library2:
statements2
Library3:
statements3
Log in to answer this question.
