This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Using while loop to go over all files in a folder

Hi,

I am currently running fastx_trimmer tool in order to trim all my paired end RNAseq input files using while loop.

The filenames are as follows:

1_1.fastq
1_2.fastq
2_1.fastq
2_2.fastq
.................
25_1.fastq
25_2.fastq

Below is the code that I use to run the while loop in a bash script using byobu screen

#!/usr/bin

i=1
while [ $i -le 25 ]
do
  cd fastq
  cd unzip_fastq
  fastx_trimmer -Q33 -f 1 -l 100 -i $i_1.fastq  -o  $i_1_trimmed.fastq
  mv $i_1_trimmed.fastq  ../trimmed_fastq
  fastx_trimmer -Q33 -f 1 -l 100 -i $i_2.fastq -o $i_2_trimmed.fastq
  mv $i_2_trimmed.fastq ../trimmed_fastq
  cd ~
  ((i++))
done

But, I got the follow error.

fastx_trimmer: failed to open input file '.fastq': No such file or directory

Any help is much appreciated. Thanks in advance

Catherine

rna-seq

Seems that files are not in correct directories else code seems correct!!

_ is a valid character in a variable name. So it thinks you are looking for a variable names $i_1_trimmed.fastq. You should instead use ${i}_1_trimmed.fastq.

In addition, you can get more informative messages about errors if you put set -ex at the top of your script.

or you could escape it with: $i\_1_trimmed.fastq...

Thank you very much!! By using ${i}, it's working!!!

3 answers

You can try the following, which is slightly simpler and you can move your files after with mv

for i in folderWithFastqFiles/*.fastq
do
  fastx_trimmer -Q33 -f 1 -l 100 -i $i -o $i.trimmed.fastq
done

even simpler: for i in folderWithFastqFiles/*.fastq ...etc

True. It's an old habit :) I edited my script to simplify it.

maybe try this one:

mkdir $2 #delete this if output directory already exists...
for f in $1/*
do
  if [ -f $f ]
  then
    fastx_trimmer -Q33 -f 1 -l 100 -i $f -o $2/$(basename $f .fastq)_trimmed.fastq
  fi
done

Call it with first argument as path to your fastq and second argumant as path where to store the trimmed files.

cd into the fastq dir and copy&paste:

for f in *.fastq; do name=$(basename "$f" .fastq); fastx_trimmer -Q33 -f 1 -l 100 -i $f -o $name.trimmed.fastq; done

Log in to answer this question.