This is a test version of Biostars. For the public version, visit https://www.biostars.org.
remove empty reads from fastq files

Hi,

Before perform STAR mapping I need to remove empty reads from my fastq files.

How can I do it without using bioawk (I cannot add it to the HPC). I have bedtools, samtools and bbmap, but I didn't find any solution in these packages.

Best, d.

rna-seq
• 4,617 views • link
updated by GenoMax GenoMax • written by dentepre • dentepre

Why can't you edit to your cluster? It is straight-forward to compile:

git clone https://github.com/lh3/bioawk.git
cd bioawk
make

or use a package manager like miniconda for it. You will have a hard time working on a HPC (or any machine) if you cannot add any software in the long-term.

• 0 views • link
written by ATpoint ATpoint

try cutadapt minimum length option or seqkit seq --min-len option

• 0 views • link
written by cpad0112 cpad0112

2 answers

With BBMap suite:

reformat.sh in=your.fq.gz out=filtered.fq.gz minlength=N

Set N to a reasonable number (very short reads are not useful anyway). Set to 1 if you just want to remove empty reads.

• 0 views • link
written by GenoMax GenoMax

how would the command look for two input files?

• 0 views • link
written by WAGNER • WAGNER

I assume you mean a paired-end dataset. For single-end data you will simply need to run this twice for the two files.

For paired-end data use

reformat.sh in1=R1.fq.gz in2=R2.fq.gz out1=R1.filtered.fq.gz out2=R2.filtered.fq.gz  minlength=N
• 0 views • link
written by GenoMax GenoMax 
gunzip -c input.fq.gz | paste - - - - | awk -F '\t' '($2!="")' | tr "\t" "\n"
• 0 views • link
written by Pierre Lindenbaum Pierre Lindenbaum

Log in to answer this question.

More posts like this