This is a test version of Biostars. For the public version, visit https://www.biostars.org.
How to execute one command on all files within a directory

Hi, I have several .fastq (R1 and R2) files in a directory. For example files: 9-Lmm05-P1413070_S9_L001_R1_001.fastq.gz, 9-Lm05-P1413070_S9_L001_R2_001.fastq.gz. I want to run the following command for the all fastq files in the directory and generate separate output directories for each pair (R1 and R2) by their prefix name (e.g. 9-Lmm05-P1413070).

$spades.py -k 21,33,55,77 -1 file1_R1_001.fastq.gz -2 file2_R2_001.fastq.gz -o Assembly

Could you please help to suggest a perl/python script to perform that.

Thank you,

python fastq perl
• 2,729 views • link
updated by asalimih • asalimih • written by Kumar Kumar

What have you tried?

• 0 views • link
written by swbarnes2 swbarnes2

I was trying as following but I think it was not correct. I see where it was wrong as asalimih suggesting.

for f in /home/R*
do
    spades.py -k 21,33,55,77 -1 $f -2 $f -o ${f/_R1*/\/Assembly}
done

Thank you for your help.

• 0 views • link
written by Kumar Kumar

1 answer

Here is a bash solution. I assume the Assembly in your example is actually the output file so I replaced it according to your explanation.

for f in *_R1_*
do
    $spades.py -k 21,33,55,77 -1 $f -2 ${f/R1/R2} -o ${f/_R1*/\/Assembly}
done

${f/R1/R2} is equivalent to replacing R1 with R2 in the name of the file.

• 0 views • link
written by asalimih • asalimih

Thank you very much for you help.

• 0 views • link
written by Kumar Kumar

Hi, I have a follow-up question. I am running this script and getting all the output files in the directory "Assembly". However, I am looking to add a prefix in the all files name in the Assembly folder according to the input fastq (sample name). For example, my sample name is Sal521_R1.fastq.gz, Sal521_R2.fastq.gz and I want to add Sal521 in all the output files in the "Assembly" folder such as Sal521_scaffolds.fasta, Sal521_contigs.fasta etc. Thank you!

• 0 views • link
written by Kumar Kumar

assuming you have a directory structure of this form:

./A
./A/Assembly/a.txt
./A/Assembly/b.txt
./B
./B/Assembly/g.txt
./B/hh.txt
./bbb.txt

The following code may do what you want:

find . -mindepth 2 -maxdepth 3 -type f -path '*Assembly*/*' | while read line 
do
   mv $line $(sed -E 's/\.\/(.+)\/Assembly\/(.*)/\.\/\1\/Assembly\/\1_\2/g' <<< "$line")
done

Output:

./A
./A/Assembly/A_a.txt
./A/Assembly/A_b.txt
./B
./B/Assembly/B_g.txt
./B/hh.txt
./bbb.txt

This code only affects the files which are in an Assembly folder and attach the name of the previous folder of the Assembly folder to each of files inside the Assembly folder.

• 0 views • link
written by asalimih • asalimih

Log in to answer this question.

More posts like this