Hi.
I'm trying to concatenate 4 files into one. This is how my raw data looks like:
> S9_L001_R1_001_1P.fq.gz
> S9_L001_R1_001_1U.fq.gz
> S9_L001_R1_001_2P.fq.gz
> S9_L001_R1_001_2U.fq.gz
> S10_L001_R1_001_1P.fq.gz
> S10_L001_R1_001_1U.fq.gz
> S10_L001_R1_001_2P.fq.gz
> S10_L001_R1_001_2U.fq.gz
I have twenty samples (S1-20) and all samples consist of four files (1P, 1U, 2P and 2U). The code I've come up with but that doesn't work looks like this:
for i in {1..20};
do for j in 1 2;
do cat S${i}_L001_R1_001_${j}*.fq.gz >S${i}_concatenate.fq.gz; done; done
It only concatenates any 2 files from each sample.
Any suggestions? Thanks.
•
written
by
Roland •
2 answers
Change the > to >> in the inner loop
>Writes a new file, overwriting anything already there>>Also creates a new file, but will append to the existing file if present
Your code only writes the two S${i}_L001_R1_001_2*.fq.gz files for any given i because it is overwriting the output of the S${i}_L001_R1_001_1*.fq.gz files in the second pass through the inner loop
for i in {1..20}; do
for j in 1 2; do
# ONLY CHANGE: ">" => ">>"
cat S${i}_L001_R1_001_${j}*.fq.gz >> S${i}_concatenate.fq.gz;
done
done
Good luck, np!
I am all for writing code to support tedious tasks, and I hope you get your answer. That said, it seems easier to type cat and paste 10 names, and do so twice, than to wait for responses here.
From what I can tell, the only thing that needs changing is * to ?
for i in {1..20};
do for j in 1 2;
do cat S${i}_L001_R1_001_${j}?.fq.gz > S${i}_concatenate.fq.gz; done; done
When in doubt, I suggest you put an echo command in front of your actual command. It will print everything on screen without executing it, so it may be easier to troubleshoot what is wrong.
for i in {1..20};
do for j in 1 2;
do echo "cat S${i}_L001_R1_001_${j}?.fq.gz > S${i}_concatenate.fq.gz" ; done; done
Maybe this will do the trick:
for i in {1..20};
do cat S${i}_L001_R1_001_??.fq.gz > S${i}_concatenate.fq.gz; done
The ? vs. ?? do?
Ah just tried it, the ? is very helpful as a wildcard - thank you
Log in to answer this question.
I hope there is a reason you are trying to cat these together. Based on the names it looks like these are properly paired and unpaired reads after trimming.
You code is ignoring the
1P, 1U, 2P and 2Uin names. What order do you want to concatenate those pieces in?Since I'm not mapping the reads to a reference genome or building my own, I figured I might as well treat them as single end reads.
I don't think it matters what order I map them in, but I guess 1P-1U-2P-2U