The Biostar Herald publishes user submitted links of bioinformatics relevance. It aims to provide a summary of interesting and relevant information you may have missed. You too can submit links here.

This edition of the Herald was brought to you by contribution from Istvan Albert, Rob, and was edited by Istvan Albert,

@wkhuber.bsky.social on Bluesky (bsky.app)

Why does anyone use the Wilcoxon test for anything? It's not even transitive. It's sensitive to things one most likely is not looking for (changes in distribution shape) and relatively insensitive to what one usually is looking for (change in location).

https://www.sciencedirect.com/science/article/pii/S0167715215302996

submitted by: Istvan Albert

Scikit-bio: a fundamental Python library for biological omic data analysis | Nature Methods (www.nature.com)

We introduce scikit-bio, a Python library for bioinformatics that is oriented toward omic data analysis. With more than 500 public-facing functions, classes and methods, scikit-bio provides a comprehensive suite of data structures and algorithms designed to address the fundamental analytical challenges and opportunities inherent to omics. Although scikit-bio supports basic sequence analysis, its true strength lies in the analysis of sample‑by‑feature tables and associated tree structures, the two data representations common to many omics studies.

submitted by: Istvan Albert

Pyranges v1: a Python framework for ultrafast sequence interval operations (www.biorxiv.org)

Sequence interval algebra is key to modern bioinformatics. Pyranges v1 offers a Python Pandas-based interface to a comprehensive palette of Rust-powered operations (e.g., overlap, count, slice intervals), enabling the intuitive development of efficient pipelines for diverse sequence data, including gene annotations, mapped reads, and protein domains. Pyranges is faster, consumes less memory, and offers more functionalities than alternative tools including BEDTools, emerging as an innovative one-stop shop for omics analysis.

It now has a rust based backend:

https://github.com/pyranges/ruranges

submitted by: Istvan Albert

@torstenseemann.bsky.social on Bluesky (bsky.app)

This is the last major release of Prokka. But don't be sad, because @oschwengers.bsky.social already has an excellent replacement called Bakta you can migrate to.

bioinformatics #microbiology #genomics

https://github.com/tseemann/prokka/releases/tag/v1.15.6

submitted by: Istvan Albert

mim: A lightweight auxiliary index to enable fast, parallel, gzipped FASTQ parsing (www.biorxiv.org)

As computers have been getting faster, and adding more cores, so too have bioinformatics software developers been working on ever more efficient lightweight methods to accurately analyze sequencing data. As we develop new methods based on ever faster methods for lightweight mapping, and sketching, etc., there is one step of the high-throughput pipeline that has basically stopped scaling altogether --- decompression and parsing.

The FASTQ format itself is relatively deoptimized for machine parsing, but the much larger problem is that the vast majority of this data is (for good reason) stored, compressed, and processed in a compressed format. For historical reasons, that compression format is gzip, a reasonably efficient but fundamentally serial decompression format. While there are methods to try to speed up decompression on many cores (e.g. rapidgzip). They perform speculative decoding and themselves end up consuming considerable compute resources. Yet, conceptually, what we'd like is trivial. If we have a 10GB input file and 10 threads, we'd like each thread to process ~1GB of the compressed input file independently of the others to perform our embarrassingly parallel task on it (e.g. read alignment). As the scale of data get ever larger, the decompression and parsing themselves become bottleneck steps.

To address this issue, we've developed mim, an auxiliary lightweight index to enable fast, parallel parsing of gzipped FASTQ files. Mim indexes a gzipped FASTQ file (a one-time process eventually designed to be done by the data curators / repositories) that creates, throughout the file, a series of checkpoints, from which compression can proceed independently and in parallel. Further, the mim index is "content aware", and so, with each checkpoint, it stores information about record boundaries and record ranks (essential for efficient paired-end parsing) in the indexed file. The index itself also incorporates several other nice features, like a cryptographic checksum of the file contents to ensure that you're using the index for the file you have, and the ability to embed arbitrary user data in the index itself.

To demonstrate the utility of this approach, we've also built mim-parser, which is a modified version of kseq++ that makes use of the mim index to enable efficient parallel decompression and parsing of FASTA/FASTQ files. We demonstrate that this provides a near-linear speedup in the number of threads being used. The index itself is quick to build (though we've not yet optimized construction), a one-time task, and small (about 1/1000-th the size of the compressed input file). Our hope is to demonstrate the utility of this approach and to build these indices for a large fraction of existing data in the major repositories (perhaps as a community effort or with the help of the repositories themselves). The index is also robust to many different types of input gzip files (single streams, multi-member archives, and even BGZF files). While we're already excited with what we're seeing from the prototype, we have a series of enhancements we hope to make including a Rust implementation and Python bindings for that Rust implementation, faster construction, even faster parsing policies, and the ability to remotely fetch existing indices using the cryptographic hash they encode.

• mim-preprint
• mim GitHub repo

submitted by: Rob

Want to get the Biostar Herald in your email? Who wouldn't? Sign up righ'ere: toggle subscription