This is a test version of Biostars. For the public version, visit https://www.biostars.org.
What is the most optimal way to count the nucleotide bases of each fastq file in my directory using UNIX commands?

I have a bunch of fastq files, and I need to write a one line UNIX command that will write the word count (wc) of how many nucleotides EACH file contains, not the total. It should look like this:

321903 1.fastq

314156 2.fastq

13515 3.fastq

...

and so on.

So far I have

cat *.fastq | awk 'NR%4 == 2 {print $0}'| tr -d '\n' | wc -c

but that doesn't work. I can't find the answer this specific anywhere.

unix fastq bash
• 7,365 views • link
updated by Alex Reynolds Alex Reynolds • written by Tom • Tom

I guess you just need to run the command on individual files, in a loop, instead of *.fastq, to get the counts per sample. Other than that I don't see anything wrong.

• 0 views • link
written by GouthamAtla GouthamAtla

I am using this command to find nucleotide sequences:

*.fastq; do echo -n ${file}; grep -o [actgnACTGN] $file | wc -l; done;

however, I am getting this error:

-bash: syntax error near unexpected token `do'

please guide how to resolve this issue

• 0 views • link
written by Fizzah • Fizzah

That is not a valid for loop. You have no "for" in it. Just copy any of the code suggestions here properly.

• 0 views • link
written by ATpoint ATpoint

3 answers

using only awk.

for F in *.fastq ; do echo -n "$F :" && awk 'NR%4 == 2 {N+=length($0);} END { printf("%d\n",N);}' $F ; done
• 0 views • link
updated by Ram Ram • written by Pierre Lindenbaum Pierre Lindenbaum

will this code work also for fastq.gz file?

• 0 views • link
written by User000 User000

could you give me some hints? thanks

• 0 views • link
written by User000 User000 
for F in *.fastq.gz; do echo -n "$F :" && zcat $F | paste - - - - | cut -f 2 | tr -d '\n'| wc -c; done >> res.txt

Something like this? A mix of your code and this one

• 0 views • link
written by User000 User000

Another option is to use Unix find:

$ find *.fastq -exec sh -c "awk 'NR%4==2' {} | tr -d '\n' | wc -c | sed -e 's/^ *//' | tr -d '\n'; echo '\t{}';" \;

Sample output:

568832   hla.example.illumina.0.1.fastq
568832   hla.example.illumina.0.2.fastq
3102624  hla.example.iontorrent.0.1.fastq
• 0 views • link
written by Alex Reynolds Alex Reynolds

can anyone describe the following command in detail:

$ find .fastq -exec sh -c "awk 'NR%4==2' {} | tr -d '\n' | wc -c | sed -e 's/^ //' | tr -d '\n'; echo '\t{}';" \;

• 0 views • link
written by Fizzah • Fizzah
  1. find looks for files that end with .fastq.
  2. On each file it finds, it runs a couple commands on that file, which are specified within two quotation marks (").
  3. That awk command takes the second line of every four lines (second line of every FASTQ record in the fastq file specified by {}), and it pipes that line to a series of additional commands:
  4. The tr command strips the newline from the second line.
  5. The wc command returns the number of characters in that line.
  6. The sed command strips space characters from the character count.
  7. The tr command strips the newline from the result from sed.
  8. The echo command reports the number of characters from wc and the filename from find.

Learning the command line is a powerful skill.

• 0 views • link
written by Alex Reynolds Alex Reynolds

You could also name specific nucleotides you are interested in directly:

for file in *.fastq; do echo -n ${file}; grep -o [actgnACTGN] $file | wc -l; done;

• 0 views • link
written by Biomonika (Noolean) Biomonika (Noolean)

Log in to answer this question.

More posts like this