Biostar Beta. Not for public use.
Question: How to combine pair end multiple fastq raw read file
0
Entering edit mode

I want to combile two files into one file:

First files name is like this:

218_9W_Pa2_R2_unmapped.fq

218_9W_Pa2_R1_unmapped.fq

218_9W_Pa1_R2_unmapped.fq

218_9W_Pa1_R1_unmapped.fq

218_9W_Co2_R2_unmapped.fq

218_9W_Co2_R1_unmapped.fq

Second file names is like this:

218_9W_Pa2_R2_mapped.fq

218_9W_Pa2_R1_mapped.fq

218_9W_Pa1_R2_mapped.fq

218_9W_Pa1_R1_mapped.fq

218_9W_Co2_R2_mapped.fq

218_9W_Co2_R1_mapped.fq

After combined both files produce file like this:

218_9W_Pa2_R2.fq

218_9W_Pa2_R1.fq

218_9W_Pa1_R2.fq

218_9W_Pa1_R1.fq

218_9W_Co2_R2.fq

218_9W_Co2_R1.fq

Actually, I want to combine mapped and unmapped raw read file to produce final set

ADD COMMENTlink 2.9 years ago Bioinfonext • 150 • updated 2.9 years ago genomax 68k
3
Entering edit mode

Shell:

for f in *_mapped.fq; do
    prefix=${f/_mapped.fq/};
    cat $f ${prefix}_unmapped.fq > $prefix.fq;
done

Or using rush (A GNU parallel like tool in Go surpporting Linux/OS X/Windows!)

ls *_mapped.fq | rush 'cat {} {.^_mapped}_unmapped.fq > {.^_mapped}.fq '

--- update ---

Much easier in rush v0.1.8+:

ls *_mapped.fq | rush -v p={.^_mapped}  'cat {} {p}_unmapped.fq > {p}.fq '
ADD COMMENTlink 2.9 years ago shenwei356 4.6k
Entering edit mode
0

Hi,

Can you please also put files location in above shell scripts and it should generate a separate directory that contain merge files.

ADD REPLYlink 2.9 years ago
Bioinfonext
• 150
Entering edit mode
0

Here's an example of redirecting content to file in a directory, dir/file, folder dir will be automatically created if not existed, you need create folder first by mkdir -p dir if it does not exit.

echo hello > dir/file
ADD REPLYlink 2.9 years ago
shenwei356
4.6k
2
Entering edit mode

GNU Parallel solutions:

parallel cat {1} {2} '>' {=1 s/_mapped// =} ::: *_mapped.fq :::+ *_unmapped.fq

parallel --xapply cat {1} {2} '>' {=1 s/_mapped// =} ::: *_mapped.fq ::: *_unmapped.fq

parallel cat {} {= s/_unmapped/_mapped/ =} '>' {= s/_unmapped// =} ::: *_unmapped.fq

Read more about GNU Parallel on Gnu Parallel - Parallelize Serial Command Line Programs Without Changing Them

ADD COMMENTlink 2.9 years ago ole.tange ♦ 3.4k
Entering edit mode
0

thanks for the examples of editing input data in GNU parallel. i did not know input data can be edited by the embeded perl expression, not just the common replacement strings. so i added a RS for removing suffix as shown in the rush examle

ADD REPLYlink 2.9 years ago
shenwei356
4.6k
Entering edit mode
0

thanks, I never used GNU parallel command. please share link to download to Cent OS platform and after download can I use any of these three command.

ADD REPLYlink 2.9 years ago
Bioinfonext
• 150
Entering edit mode
0

Please just google GNU parallel or install via yum.

ADD REPLYlink 2.9 years ago
shenwei356
4.6k
Entering edit mode
0

If I google GNU parallel, it will take me to this page:

https://www.gnu.org/software/parallel/

Which One should I DOWNLOAD for Cent OS platform?

ADD REPLYlink 2.9 years ago
Bioinfonext
• 150
Entering edit mode
1

See https://www.gnu.org/software/parallel/ under "Community maintained packages".

Or use'the 10 second installation:

(wget -O - pi.dk/3 || curl pi.dk/3/ || fetch -o - http://pi.dk/3) | bash

See more on: Gnu Parallel - Parallelize Serial Command Line Programs Without Changing Them

ADD REPLYlink 2.9 years ago
ole.tange
♦ 3.4k
Entering edit mode
0

Forget GNU parallel if you think it hard to install, use rush instead. rush provides executable binary files for Linux/OS X/Windows. Just download the binary files, decompress and immediately run

Much easier in rush v0.1.8+:

ls *_mapped.fq | rush -v p='{.^_mapped}'  'cat {} {p}_unmapped.fq > outdir/{p}.fq '
ADD REPLYlink 2.9 years ago
shenwei356
4.6k
Entering edit mode
0

Thank you very much

can you please also share a copy linux command: to coy a directory content into a desired directory.

I use this command: cp -r dir1 dir2

but it create a new directory (dir1) within dir2. I need to only copy contents of dir1 into dir2

ADD REPLYlink 2.9 years ago
Bioinfonext
• 150
Entering edit mode
1

Try cp dir1/* dir2

ADD REPLYlink 2.9 years ago
WouterDeCoster
39k
Entering edit mode
1

I think you need read some shell tutorial, or you're going to be frequently stuck by these simple operations.

ADD REPLYlink 2.9 years ago
shenwei356
4.6k
Entering edit mode
0

yes...surely I will go through the shell tutorial.

ADD REPLYlink 2.9 years ago
Bioinfonext
• 150

Login before adding your answer.

Similar Posts
Loading Similar Posts
Powered by the version 2.0