This is a test version of Biostars. For the public version, visit https://www.biostars.org.
How to count fasta sequences efficiently using (or not ) biopython

This is not a very memory friendly way of counting sequences from a multi fasta, any ideas to improve this?

generator = SeqIO.parse("test_fasta.fasta","fasta")
sizes = [len(rec) for rec in SeqIO.parse("test_fasta.fasta", "fasta")]

I'm avoiding using tools like grep since I want to make this more portable

biopython python fasta
• 25,046 views • link
updated by Mark Mark • written by juan.crescente • juan.crescente

How to cont fasta

Count/concatenate/check length?

• 0 views • link
written by GenoMax GenoMax

Count as in the description, also could have guessed from the example. PD title updated

• 0 views • link
written by juan.crescente • juan.crescente

bioawk? github.com/lh3/bioawk

• 0 views • link
written by Ram Ram

6 answers

Standard Python will be faster than BioPython:

fh = open("test_fasta.fasta")
n = 0
for line in fh:
    if line.startswith(">"):
        n += 1
fh.close()

or shorter and possibly faster:

num = len([1 for line in open("test_fasta.fasta") if line.startswith(">")])
• 0 views • link
written by Andrzej Zielezinski Andrzej Zielezinski

Count the number of sequences in 1 fasta file:

grep -c ">" file.fasta

Count the number of sequences in several fasta files:

find /home/folder/to/file/ -name "*.fasta" | parallel grep -Hc ">" {}

Get the length of each instance of a fasta file:

grep -v ">" 180110.fasta | awk '{print length}'
• 0 views • link
written by tiago211287 tiago211287

You could take a look at this python lib: https://github.com/mdshw5/pyfaidx. It makes use of .fai files or may generate one. It is also compatible with gziped fasta's.

• 0 views • link
written by yhoogstrate yhoogstrate

Using grep in Python

import subprocess 
subprocess.call(['grep', '-c', '>', 'test_fasta.fasta'])
• 0 views • link
written by Carlos Caicedo Carlos Caicedo

I think using pysam is the fastest way to read a FASTA/FASTQ file - especially large files. You can use it to read the FASTA file and count the number of sequences - https://onestopdataanalysis.com/read-fasta-file-python/

• 0 views • link
written by onestop_data onestop_data

According to this SO post: https://stackoverflow.com/questions/393053/length-of-generator-output

This is the fastest:

len(list(generator))

This assumes that each fasta sequence is a record, so counting the records should equal the number of sequences. One issue is that this method will consume your generator.

• 0 views • link
written by Mark Mark

Log in to answer this question.

More posts like this