This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Tool: A python library for reading and writing sequences in fasta and fastq format.

Hello, erveyone, I write a c library named seqio for writing and reading sequence in fasta and fastq format. I have make it open on Github https://github.com/openbioseq/fastseqio. If you have any needs for writing/reading fasta/fastq file, I strongly recommand you have a try.

seqio
• 2,682 views • link
updated by Istvan Albert Istvan Albert • written by dwpeng • dwpeng

Do you want to say a little bit about _why_ they should try out this library, rather than something like kseq, kseqpp, etc.?

• 0 views • link
written by Rob Rob

Here are a few reasons why I recommend using it:

1. No macro definition required

#include "seqio.h"
#include <stdio.h>

int
main(int argc, char* argv[])
{
  if (argc == 1) {
    fprintf(stderr, "Usage: %s <in.fasta>\n", argv[0]);
    return 1;
  }
  // Step1: set open options
  seqioOpenOptions openOptions = {
    .filename = argv[1],
  };
  // Step2: open file
  seqioFile* sf = seqioOpen(&openOptions);
  // step3: read records
  seqioRecord* record = NULL;
  // step4: read records one by one
  while ((record = seqioRead(sf, record)) != NULL) {
    // setp5: do something with the record
    printf("name: %s: %lu\n", record->name->data, record->sequence->length);
    // !!! Do not free record, 
    // !!! beacuse it will be freed by seqioRead automatically.
  }
  // step6: close file
  seqioClose(sf);
}

You only need to include the header file to use it, without the need for macro definitions, as shown below

2. No need to manually release memory

SeqioRead will automatically release memory, so you don't need to manually release memory to avoid memory leaks.

3. Intuitive API

Using it is like using a regular file, only opening, reading, and closing are required, without the need for internal implementation of relationships.

/**
  * @brief open a file
  * @param options open options
  * @return seqioFile* file
 */
seqioFile* seqioOpen(seqioOpenOptions* options);

/**
  * @brief read a record
  * @param file
  * @param record
  * @return seqioRecord* record or NULL if the file is end
 */
seqioRecord* seqioRead(seqioFile* file, seqioRecord* record);

/**
  * @brief write a fasta record
  * @param file
  * @param record
  * @param options
 */
void seqioWriteFasta(seqioFile* sf, seqioRecord* record, seqioWriteOptions* options);


/**
  * @brief close a file
  * @param file
 */
void seqioClose(seqioFile* file);

4. Support read and write of compressed files

Support reading and writing compressed files, you don't need to worry about whether the file is compressed or uncompressed, just set the parameters.

typedef struct {
  char* filename;   // filename
  bool isGzipped;   // it will be detected automatically if mode is seqOpenModeRead
  seqOpenMode mode; // default is seqOpenModeRead
} seqioOpenOptions;

typedef struct {
  size_t lineWidth;      // fasta file line width (default: 0, no wrap)
  bool includeComment;   // include comment in fasta record (default: true)
  baseCase baseCase;     // base case (default: original)
} seqioWriteOptions;

5. Both C and C++ can use it

• 0 views • link
written by dwpeng • dwpeng

I would recommend you provide Python binding.

• 0 views • link
written by MatthewP MatthewP

Great suggestion, I will try to implement it.

• 0 views • link
written by dwpeng • dwpeng

Hi, I have implemented this python library.

• 0 views • link
written by dwpeng • dwpeng

Hello, everyone. I built a python library named fastseqio on top of seqio. It supports to write and read fasta/fastq files. Very fast parse tool.

import os

from fastseqio import seqioFile, Record


def test_read():
    file = seqioFile("test-data/test2.fa")

    records = []
    for record in file:
        records.append(record)

    assert len(records) == 3

    assert records[0].name == "a"
    assert records[1].name == "b"
    assert records[2].name == "c"

    records = []
    for record in file:
        records.append(record)

    assert len(records) == 0

    file.reset()

    records = []
    for record in file:
        records.append(record)

    assert len(records) == 3


def test_write():
    file = seqioFile("out.fa", "w")

    file.writeFasta("test", "ACGGGGGGGTTTT")
    file.writeFasta("test", "ACGGGGGGGTTTT")

    file.close()

    content = ">test\nACGGGGGGGTTTT\n>test\nACGGGGGGGTTTT\n"

    with open("out.fa", "r") as fp:
        data = fp.read()
        assert data == content

    os.remove("out.fa")


def test_write_gz():
    # compress by extension or let `compressed=True`
    file = seqioFile("out.fa.gz", "w")

    file.writeFasta("test", "ACGGGGGGGTTTT")
    file.writeFasta("test", "ACGGGGGGGTTTT")

    file.close()
    import gzip

    content = ">test\nACGGGGGGGTTTT\n>test\nACGGGGGGGTTTT\n"

    with open("out.fa.gz", "rb") as fp:
        data = fp.read()
        data = gzip.decompress(data).decode("utf-8")
        assert data == content

    os.remove("out.fa.gz")


def test_record():
    record = Record("test", "ACGGGGGGGTTTT")

    assert record.name == "test"
    assert record.sequence == "ACGGGGGGGTTTT"

    record.name = "test2"
    record.sequence = "ACGGGGGGGTTTTTTTT"

    assert record.name == "test2"
    assert record.sequence == "ACGGGGGGGTTTTTTTT"

    hpc = record.hpc_commpress()
    assert hpc == "ACGT"

    rev = record.reverse()
    assert rev == "TTTTTTTTGGGGGGGCA"

    length = record.length()
    assert length == 17

    length = len(record)
    assert length == 17

    record.sequence += "xxx"
    assert record.length() == 20
    assert record.sequence == "ACGGGGGGGTTTTTTTTxxx"
    assert len(record) == 20

    record.sequence = "ACGGGGGGGTTTT"

    sub = record.subseq(2, 5)
    assert sub == "GGGGG"


def test_kmers():
    record = Record("test", "ACGGGG")

    kmers = list(record.kmers(4))
    assert len(kmers) == (len(record) - 4 + 1)
    assert kmers == ["ACGG", "CGGG", "GGGG"]
• 0 views • link
written by dwpeng • dwpeng

1 answer

Two years ago, I published fastseqio. This is a python library that written in C. Now It provides all platform support. If you want use a light library to parse fasta/fastq file, it will be a good choice. You can find the documentation at https://openbioseq.github.io/fastseqio/ To be honest, biopython is too heavy for only parsing fasta/fastq file.

pip install fastseqio
• 0 views • link
written by dwpeng • dwpeng

In all fairness, BioPython may be slow for parsing FASTQ records because it decodes the FASTQ qualities into a list of integers.

To the best of my understanding, your library does not decode the FASTQ qualities

If all one needs is to read in four-line records from FASTQ format, then the fastest and simplest Python implementation I can think of would be something like:

import sys

stream = sys.stdin

for name in stream:
    name = name.strip()[1:]
    seq = next(stream).strip()
    tmp = next(stream).strip()
    qual = next(stream).strip()
    print(name, len(seq))

An iterator requires no installation, has no memory overhead, and is likely very fast, since it uses well-optimized internal implementations.

You should benchmark your solution relative to comparable solutions, rather than against a solution that performs a much more difficult task.

• 0 views • link
written by Istvan Albert Istvan Albert

Log in to answer this question.

More posts like this