This is a test version of Biostars. For the public version, visit https://www.biostars.org.
How to combine pair end multiple fastq raw read file

I want to combile two files into one file:

First files name is like this:

218_9W_Pa2_R2_unmapped.fq

218_9W_Pa2_R1_unmapped.fq

218_9W_Pa1_R2_unmapped.fq

218_9W_Pa1_R1_unmapped.fq

218_9W_Co2_R2_unmapped.fq

218_9W_Co2_R1_unmapped.fq

Second file names is like this:

218_9W_Pa2_R2_mapped.fq

218_9W_Pa2_R1_mapped.fq

218_9W_Pa1_R2_mapped.fq

218_9W_Pa1_R1_mapped.fq

218_9W_Co2_R2_mapped.fq

218_9W_Co2_R1_mapped.fq

After combined both files produce file like this:

218_9W_Pa2_R2.fq

218_9W_Pa2_R1.fq

218_9W_Pa1_R2.fq

218_9W_Pa1_R1.fq

218_9W_Co2_R2.fq

218_9W_Co2_R1.fq

Actually, I want to combine mapped and unmapped raw read file to produce final set

rna-seq
• 5,269 views • link
updated by GenoMax 16K • | written by Bioinfonext 48

2 answers

Shell:

for f in *_mapped.fq; do
    prefix=${f/_mapped.fq/};
    cat $f ${prefix}_unmapped.fq > $prefix.fq;
done

Or using rush (A GNU parallel like tool in Go surpporting Linux/OS X/Windows!)

ls *_mapped.fq | rush 'cat {} {.^_mapped}_unmapped.fq > {.^_mapped}.fq '

--- update ---

Much easier in rush v0.1.8+:

ls *_mapped.fq | rush -v p={.^_mapped}  'cat {} {p}_unmapped.fq > {p}.fq '
• 0 views • link
written by shenwei356 870

Hi,

Can you please also put files location in above shell scripts and it should generate a separate directory that contain merge files.

• 0 views • link
written by Bioinfonext 48

Here's an example of redirecting content to file in a directory, dir/file, folder dir will be automatically created if not existed, you need create folder first by mkdir -p dir if it does not exit.

echo hello > dir/file
• 0 views • link
written by shenwei356 870

GNU Parallel solutions:

parallel cat {1} {2} '>' {=1 s/_mapped// =} ::: *_mapped.fq :::+ *_unmapped.fq

parallel --xapply cat {1} {2} '>' {=1 s/_mapped// =} ::: *_mapped.fq ::: *_unmapped.fq

parallel cat {} {= s/_unmapped/_mapped/ =} '>' {= s/_unmapped// =} ::: *_unmapped.fq

Read more about GNU Parallel on Gnu Parallel - Parallelize Serial Command Line Programs Without Changing Them

• 0 views • link
written by ole.tange 448

thanks for the examples of editing input data in GNU parallel. i did not know input data can be edited by the embeded perl expression, not just the common replacement strings. so i added a RS for removing suffix as shown in the rush examle

• 0 views • link
written by shenwei356 870

thanks, I never used GNU parallel command. please share link to download to Cent OS platform and after download can I use any of these three command.

• 0 views • link
written by Bioinfonext 48

Please just google GNU parallel or install via yum.

• 0 views • link
written by shenwei356 870

Forget GNU parallel if you think it hard to install, use rush instead. rush provides executable binary files for Linux/OS X/Windows. Just download the binary files, decompress and immediately run

Much easier in rush v0.1.8+:

ls *_mapped.fq | rush -v p='{.^_mapped}'  'cat {} {p}_unmapped.fq > outdir/{p}.fq '
• 0 views • link
written by shenwei356 870

Thank you very much

can you please also share a copy linux command: to coy a directory content into a desired directory.

I use this command: cp -r dir1 dir2

but it create a new directory (dir1) within dir2. I need to only copy contents of dir1 into dir2

• 0 views • link
written by Bioinfonext 48

I think you need read some shell tutorial, or you're going to be frequently stuck by these simple operations.

• 0 views • link
written by shenwei356 870

yes...surely I will go through the shell tutorial.

• 0 views • link
written by Bioinfonext 48

Log in to answer this question.

More posts like this

No similar posts found.