fastx_reverse_complement truncating files?

Dear all,

I am using FASTX-Toolkit 0.0.14-foss-2016a and using the command line:

$ module load FASTX-Toolkit; fastx_reverse_complement -z -i [input.fastq] -o [output.fastq.gz]

When I use:

$ wc -l  input.fastq.gz

I get 6191703 returned vs. 4742402 for the length of the output.fastq.gz

Suggestions please?

fastx reverse compliment rna-seq

• 2,634 views • link

updated 7.5 years ago by ATpoint

• written 7.5 years ago by miyagi •

If the output is really a fastq.gz you have to uncompress it to count the lines:

$ zcat input.fastq.gz | wc -l

Same for input file of course.

• 0 views • link

written 7.5 years ago by finswimmer

hi thanks for the reply, the input file is a fastq, however i'm comparing the length to the original fastq.gz file because i know that the fastq file and the compressed file will have different lengths.

• 0 views • link

written 7.5 years ago by miyagi •

however i'm comparing the length to the original fastq.gz

How do you do that?

Please use the ADD REPLY below a post you like to reply to.

Thanks!

• 0 views • link

written 7.5 years ago by finswimmer

I just use $ wc -l input.fastq.gz and compare that to $ wc -l output.fastq.gz

• 0 views • link

written 7.5 years ago by miyagi •

regardless, the GB of the files goes from being 1.15 for the original file to 0.976 to the new file even though both are compressed

• 0 views • link

written 7.5 years ago by miyagi •

and i'm getting an error indicating the file is corrupted when using a downstream software analysis

• 0 views • link

written 7.5 years ago by miyagi •

Fastx is very old and in low (or no) maintanance. Use seqtk and see if it performs better. Also, you cannot use wc -l on gzipped files. You have to decompress first, like zcat file.fq.gz | wc -l. Probably the differences you see are due to the improper use of wc on a compressed file. Fastx, even though being old should be able to do a simple revcomp properly.

• 0 views • link

written 7.5 years ago by ATpoint

Hey, thank you! haha .I know.. you'd think this very simple thing would work. The worst part is not even getting a proper error. I understand what you're saying about the wc -l, though that would not explain the difference in size of the files right (0.976 GB vs 1.15). I'll give seqtk a shot and update if it works.

• 0 views • link

written 7.5 years ago by miyagi •

You should never use file sizes as a metric for anything. Files compress differently when the individual nucleotides gets rearranged, which will certainly happen when you RC them.

Use reformat.sh from BBMap suite to do the reverse complementing. Something like:

reformat.sh in=your_file.fq.gz out=rcomp_fq.gz rcomp=t

rcomp=f                 (rc) Reverse-compliment reads.
rcompmate=f             (rcm) Reverse-compliment read 2 only.

• 0 views • link

written 7.5 years ago by GenoMax

0 answers

No answers yet.