This is a test version of Biostars. For the public version, visit https://www.biostars.org.
bowtie2 shell script

Dear All,

I have tons of folders inside which I have fastq files, and these are single end reads.folders files I want to write a shell/bash script to automate the alignment.

for i in /media/usr/Elements/name/Extracted_SB_reads/C1DNA.extrSB/*.fq 
do 

bowtie2 --sensitive-local -p8 ${i} -x /media/usr/Elements/usr/bowtie2_index/hg19/ -U ${i}.fq -S $i.sam

done

but when I run the script it says:

(ERR): bowtie2-align exited with value 1
Extra parameter(s) specified: "/media/usr/Elements/usr/Extracted_SB_reads/C1DNA.extrSB/V300054326_L4_B5GHUMvudRAABGAAA-556.SB.fq"
Error: Encountered internal Bowtie 2 exception (#1)
Command: /home/usr/miniconda3/bin/bowtie2-align-s --wrapper basic-0 --sensitive-local -p8 -x /media/usr/Elements/usr/bowtie2_index/hg19/ -S /media/usr/Elements/usr/Extracted_SB_reads/C1DNA.extrSB/V300054326_L4_B5GHUMvudRAABGAAA-556.SB.fq.sam -U /media/usr/Elements/usr/Extracted_SB_reads/C1DNA.extrSB/V300054326_L4_B5GHUMvudRAABGAAA-556.SB.fq.fq /media/usr/Elements/usr/Extracted_SB_reads/C1DNA.extrSB/V300054326_L4_B5GHUMvudRAABGAAA-556.SB.fq 
(ERR): bowtie2-align exited with value 1

I am clueless . Can someone help. regards.

bowtie2 shellscript
• 3,456 views • link
updated by ATpoint ATpoint • written by amitpande74 • amitpande74

Do you have your bowtie2 indices for all the reads files already? Or do you want to map them against the same index? Could you show us the output of ls on the directory where your reads are contained?

• 0 views • link
written by Dunois • Dunois

These are reads extracted for a transposon and guideRNA. They have to be aligned to the human genome to determine the genomic location of their insertion.

(base) amit@amit-X705UDR:/media/amit/Elements/usr/Extracted_SB_reads$ ls
B2DNA_extractedSB  DNA27.extractedSB  DNA4.extractedSB    H1DNA.extractedSB  H8DNA.extractedSB   MH2DNA.extractedSB  W2DNA.extractedSB
C1DNA.extrSB       DNA28_extractedSB  DNA9_extractedSB    H2DNA.extractedSB  H9DNA.extractedSB   MH4DNA.extractedSB  W5DNA.extractedSB
DN2.extractedSB    DNA29.extractedSB  F24DNA.extractedSB  H3DNA.extractedSB  M1DNA.extratedSB    MH5DNA.extractedSB  W6DNA.extractedSB
DNA17_extractedSB  DNA30.extractedSB  H10DNA.extractedSB  H4DNA.extractedSB  M4DNA.extractedSB   MH6DNA.extractedSB
DNA18.extractedSB  DNA31.extractedSB  H11DNA.extractedSB  H5DNA.extractedSB  M6DNA.extractedSB   MH8DNA.extractedSB
DNA22.extSB        DNA32.extractedSB  H12DNA.extractedSB  H6DNA.extractedSB  M7DNA.extractedSB   MH9DNA.extractedSB
DNA26_extractedSB  DNA3.extractedSB   H13DNA.extractedSB  H7DNA.extractedSB  MH1DNA.extractedSB  W1DNA.extractedSB

(base) amit@amit-X705UDR:/media/amit/Elements/usr/Extracted_SB_reads/C1DNA.extrSB$ ls
V300054326_L3_B5GHUMvudRAABGAAA-549.SB2.fq  V300054326_L3_B5GHUMvudRAABGAAA-554.SB.fq   V300054326_L4_B5GHUMvudRAABGAAA-552.SB2.fq
V300054326_L3_B5GHUMvudRAABGAAA-549.SB.fq   V300054326_L3_B5GHUMvudRAABGAAA-555.SB2.fq  V300054326_L4_B5GHUMvudRAABGAAA-552.SB.fq
V300054326_L3_B5GHUMvudRAABGAAA-550.SB2.fq  V300054326_L3_B5GHUMvudRAABGAAA-555.SB.fq   V300054326_L4_B5GHUMvudRAABGAAA-553.SB2.fq
V300054326_L3_B5GHUMvudRAABGAAA-550.SB.fq   V300054326_L3_B5GHUMvudRAABGAAA-556.SB2.fq  V300054326_L4_B5GHUMvudRAABGAAA-553.SB.fq
V300054326_L3_B5GHUMvudRAABGAAA-551.SB2.fq  V300054326_L3_B5GHUMvudRAABGAAA-556.SB.fq   V300054326_L4_B5GHUMvudRAABGAAA-554.SB2.fq
V300054326_L3_B5GHUMvudRAABGAAA-551.SB.fq   V300054326_L4_B5GHUMvudRAABGAAA-549.SB2.fq  V300054326_L4_B5GHUMvudRAABGAAA-554.SB.fq
V300054326_L3_B5GHUMvudRAABGAAA-552.SB2.fq  V300054326_L4_B5GHUMvudRAABGAAA-549.SB.fq   V300054326_L4_B5GHUMvudRAABGAAA-555.SB2.fq
V300054326_L3_B5GHUMvudRAABGAAA-552.SB.fq   V300054326_L4_B5GHUMvudRAABGAAA-550.SB2.fq  V300054326_L4_B5GHUMvudRAABGAAA-555.SB.fq
V300054326_L3_B5GHUMvudRAABGAAA-553.SB2.fq  V300054326_L4_B5GHUMvudRAABGAAA-550.SB.fq   V300054326_L4_B5GHUMvudRAABGAAA-556.SB2.fq
V300054326_L3_B5GHUMvudRAABGAAA-553.SB.fq   V300054326_L4_B5GHUMvudRAABGAAA-551.SB2.fq  V300054326_L4_B5GHUMvudRAABGAAA-556.SB.fq
V300054326_L3_B5GHUMvudRAABGAAA-554.SB2.fq  V300054326_L4_B5GHUMvudRAABGAAA-551.SB.fq
• 0 views • link
written by amitpande74 • amitpande74

2 answers

Remove the ${i} after -p8, it is misplaced there. Pretty sure that -x path is also wrong because it has to point to the prefix of the index files, right now it points to a directory.

• 0 views • link
written by ATpoint ATpoint

I did.

for i in /media/usr/Elements/usr/Extracted_SB_reads/C1DNA.extrSB/*.fq 
do 

bowtie2 --sensitive-local -p8 -x /media/usr/Elements/usr/bowtie2_index/hg19/ -U ${i}.fq -S $i.sam

done

and now it says:

(ERR): bowtie2-align exited with value 1
stat: Bad file descriptor
Warning: Could not open read file "/media/usr/Elements/usr/Extracted_SB_reads/C1DNA.extrSB/V300054326_L4_B5GHUMvudRAABGAAA-556.SB.fq.fq" for reading; skipping...
Error: No input read files were valid
(ERR): bowtie2-align exited with value 1

actually there are 2 file extensions inside the folder .SB.fq and .SB2.fq how to include both the extensions ?

• 0 views • link
written by amitpande74 • amitpande74

When /media/usr/Elements/usr/Extracted_SB_reads/C1DNA.extrSB/*.fq is your looping $i then each $i already has the .fq suffix. ${i}.fq is redundant, -U must just be -U ${i}, and -S should be -S ${i%.fq}.sam while the latter is just my personal preference, the -U is the cause of error, next error that comes up is probably -x as it still point to a folder rather than the index prefix.

• 0 views • link
written by ATpoint ATpoint

Thank you.

I haven't tested this, but something like this would probably work:

#!/bin/bash -ex

MYPATH="/media/amit/Elements/usr/Extracted_SB_reads";
find ${MYPATH} -maxdepth 3 -type f -name "*.fq$" > ${MYPATH}/fq_names.txt;


while read INFILE; 
do

bowtie2 -p 8 -x /media/usr/Elements/usr/bowtie2_index/hg19/BT2INDEXNAME -U ${INFILE} -S ${INFILE%.fq}.sam;

done < ${MYPATH}/fq_names.txt;

Do note the /media/usr/Elements/usr/bowtie2_index/hg19/BT2INDEXNAME in there indicating that you need to replace BT2INDEXNAME with the actual name that's sitting in /media/usr/Elements/usr/bowtie2_index/hg19/. If there are files called myhumangenomebt2.1.bt2 and myhumangenome.fq.2.bt2 in /media/usr/Elements/usr/bowtie2_index/hg19/, for example, then /media/usr/Elements/usr/bowtie2_index/hg19/BT2INDEXNAME should be replaced with /media/usr/Elements/usr/bowtie2_index/hg19/myhumangenome.fq.

Are you really going to run all of this on a laptop though? It'll probably take for ever, and submitting so many tasks like this through a for loop isn't exactly efficient.

Note: the post has been edited to incorporate corrections discussed in the comments below.

• 0 views • link
written by Dunois • Dunois

Hi, Thanks for your help. These fastq files are not very big, as only those reads which have the transposon and the guide RNA have been extracted (wgs). I tried running it script though, and it gave an error:

(base) usr@usr-X705UDR:/media/usr/Elements/usr$ ./bowtie.sh 
+ MYPATH=/media/usr/Elements/usr/Extracted_SB_read
+ find /media/usr/Elements/usr/Extracted_SB_read -maxdepth 3 -type -f name '*.fq$'
./bowtie.sh: line 4: /media/usr/Elements/usr/Extracted_SB_read/fq_names.txt: No such file or directory

regards,

• 0 views • link
written by amitpande74 • amitpande74

Ah, looks like there's a typo there, could you please edit MYPATH=MYPATH=/media/usr/Elements/usr/Extracted_SB_read to MYPATH=MYPATH=/media/usr/Elements/usr/Extracted_SB_reads? Didn't catch the missing s at the end of read there.

• 0 views • link
written by Dunois • Dunois 
(base) usr@usr-X705UDR:/media/usr/Elements/usr$ ./bowtie.sh 
+ MYPATH=/media/usr/Elements/usr/Extracted_SB_reads
+ find /media/usr/Elements/usr/Extracted_SB_reads -maxdepth 3 -type -f name '*.fq$'
find: Unknown argument to -type: -

Stops here.

So corrected it to:

#!/bin/bash -ex

MYPATH="/media/usr/Elements/usr/Extracted_SB_reads";
find ${MYPATH} -maxdepth 3 -type f -name "*.fq" > ${MYPATH}/fq_names.txt;


while read INFILE; 
do

bowtie2 -p 8 -x /media/usr/Elements/usr/hg19 -U ${INFILE} -S ${INFILE%.fq}.sam;

done < ${MYPATH}/fq_names.txt;

and now it works. Thanks a lot.

• 0 views • link
written by amitpande74 • amitpande74

Ah super sorry I didn't catch that one. Wrote this on the fly, and I didn't test it, so I had no idea where the errors were. Good on you for finding it!!

I was glad to be able to help!!

• 0 views • link
written by Dunois • Dunois

Dunois : Please edit/apply all corrections to your parent comment (which I have moved to an answer). That way it would be fully correct answer.

amitpande74 : You can accept this answer (green check mark).

• 0 views • link
written by GenoMax GenoMax

Thank you GenoMax (also for moving the comment to its own answer), I've updated the OP as you've suggested.

• 0 views • link
written by Dunois • Dunois

Log in to answer this question.

More posts like this