This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Tool: RustQC: 60x speedup in RNA-seq quality control steps

Hi folks,

I just released RustQC - a Rust rewrite of a suite of different RNA-seq quality control tools ( dupRadar, featureCounts, RSeQC, Qualimap, preseq, and samtools). It generates all the same outputs in a single run, reading the BAM file only once - the end result is that it runs in 15 minutes instead of 15 _hours_, a >60x speed up (and far less disk I/O).

I wrote a blog post about how I built it here: https://seqera.io/blog/rustqc/ - TLDR is that it's written using AI, and I think that such rewrites are going to get a lot more common, fast. To try to get ahead of this with some best practices, we're also launching rewrites.bio - a set of principles to try to guide AI-generated tool rewrites. Hopefully it's a useful reference for people and gets the discussion started!

Let me know what you think, keen to get feedback on this. And I hope you find it useful in your work!

Phil

rustqc nf-core
• 1,407 views • link
updated by jared.andrews07 jared.andrews07 • written by Phil Ewels Phil Ewels

Who is going to take care of porting updates/bug fixes that happen in the original software. Would there be regular updated releases of RustQC to keep up with those changes?

It may become a headache to figure out which version of a specific program was used in the analyses people publish without digging through RustQC release notes. Is there a file that lists this information for each release of RustQC.

• 0 views • link
written by GenoMax GenoMax

Great points! The intention is that I'll keep maintaining it and updating it (plus any other contributors). I've pinned the exact versions in the docs already but you're right, it might be smart to have these bundled in release notes or something as they change over time.

I'd like to look into how we can pass this info through to reporting / final results too. For example, it could be in an output file saved by RustQC at run time. And maybe I can be clever with how outputs are versioned in MultiQC reports? "Produced by RustQC v0.1.2, based on outputs from Qualimap v2.3" or something.

Open to suggestions on how to do this well! This would be a good topic for rewrites.bio as well.

• 0 views • link
written by Phil Ewels Phil Ewels

"Produced by RustQC v0.1.2, based on outputs from Qualimap v2.3" or something.

It would be foolproof to store such text in a file in output folder(s). You could also request users to add that information in their methods section, if they cite RustQC. If there are any plots, then the text could be embedded as a footnote.

Would it be possible to allow users to select the tools to run on the command line as optional arguments? If no argument is provided that means run all tools.

• 0 views • link
written by GenoMax GenoMax

Made an issue about the output file with versions: https://github.com/seqeralabs/RustQC/issues/75

There are a few --skip-x CLI flags already to omit certain parts of the RustQC analysis. I think that everything can be toggled on and off via the config file. Could probably expand the CLI to have toggles for each tool if that'd be useful.

• 0 views • link
written by Phil Ewels Phil Ewels

Perhaps adding --skip app1,app2 option may be more versatile. On clusters (where RustQC may be installed as a module) the config may not be editable by normal users, unless it is read from local directory/user's $HOME.

• 0 views • link
written by GenoMax GenoMax

This work (along with edgePython, etc) have generated a lot of internal discussion. https://rewrites.bio/ spurred some passionate debate, particularly the 3.1 "think big" point, which was generally felt to encourage "shortcuts" rather than holistic ports that are more easily maintained and/or rolled in upstream in the original implementation. This relates to some degree to what GenoMax mentions in regard to cherry-picking functionality and general maintenance/provenance concerns.

We agreed it'd be better to avoid combinatorial merging of features, keeping a port a full reimplementation (perhaps with optional streamlining capabilities), and then building any additional tooling as a separate entity on top as needed. E.g., rewrites of dupRadar, featureCounts, etc, with RustQC then using those ports as it pleases. This would generally simplify maintenance and be more broadly useful in our opinion, though we recognize this is more work and a less direct route to the ultimate goal in this case. That said, we feel the rewrites.bio guidelines should consider reflecting a more holistic approach like that for the good of the community at large.

• 0 views • link
written by jared.andrews07 jared.andrews07

Hi jared.andrews07, apologies for the delay - biostars email went to spam.

I understand where you're coming from and have thought about this approach myself quite a bit, but I don't entirely agree. There are a few points to this:

  • A partial rewrite is often a _lot_ less work than a full rewrite. If the aim is to replace usage within a certain context (as RustQC was: for the nf-core/rnaseq pipeline), then adding the additional burden of work, validation and maintenance is problematic. Personally I'd rather have a smaller footprint which is better written / maintained. RustQC almost certainly would never have happened / been released if this was a requirement.
  • Separate tooling, even with streaming, cannot get to the kinds of 60x speed-up that I got with RustQC. That only worked with all the various counting algorithms being called _within_ the "hot loop" going over every read. You can't get the degree of parallelisation and optimisation without being in the same codebase. It might be possible with different packages in Rust sharing libraries, this is something I looked into. But it'd need such a high degree of coordination that I felt it was too brittle.
  • RustQC doesn't aim to replace the tools it emulates. It's not meant to be a full port. It's aiming to be a faithful replication of a pipeline subworkflow. Currently, anyway.

For me, the key to this is understanding that there are different types of rewrites (and this is something I intend to update rewrites.bio with when I get a moment). RustQC was a combinatorial rewrite - it made sense that way for those tools and that usage. The same will not be true for all tools.

Phil

• 0 views • link
written by Phil Ewels Phil Ewels

Fair points. Perhaps semantics, but this point:

Separate tooling, even with streaming, cannot get to the kinds of 60x speed-up that I got with RustQC. That only worked with all the various counting algorithms being called _within_ the "hot loop" going over every read. You can't get the degree of parallelisation and optimisation without being in the same codebase. It might be possible with different packages in Rust sharing libraries, this is something I looked into. But it'd need such a high degree of coordination that I felt it was too brittle.

is really key and perhaps indicates being more explicit that it's a reimplementation of specific functionality from various packages rather than a true re-write of said packages may be more accurate. But again, semantics.

From our internal discussions, one of our main concerns was a proliferation of effectively unmaintained re-writes cherry picking functionality to replace key components of whatever the author(s) cared about. So instead of one performant implementation of a tool offering all functionality, you get a half dozen cobbled together toolkits designed for effectiveness in very specific contexts. I am not sure this is something to really encourage generally. RustQC is an interesting case given how heavily the nf-core pipelines are used and how much of a difference the re-write makes. I don't know where one would really want to line drawn, per se.

I recognize the work involved, and the point in the quote is an important one in terms of performance. Motivation and context for the re-write is pretty key.

I think I also just disagree a bit, but that's okay.

• 0 views • link
written by jared.andrews07 jared.andrews07

2 answers

Hi Phil, commenting here after your response in the Nextflow channel.

I think a methods paper on RustQC and its validation would really help with academic adoption. For many users, strong benchmarks are not quite enough on their own. Having something directly citable, together with clear provenance showing which original tool versions each RustQC release corresponds to, would make published use much easier.

Speaking just from my own situation as a PhD student and the sole bioinformatics person in a mixed wet-dry lab, I know I would have a hard time convincing my PI to adopt something that is not yet published, even if the technical case looks strong. I suspect a lot of potential users are in a similar position.

• 0 views • link
written by Kinley • Kinley

I think a methods paper on RustQC and its validation would really help with academic adoption.

That may be a tall order .. specially for an AI assisted rewrite of existing tools that have been published on their own already.

I know I would have a hard time convincing my PI to adopt something that is not yet published, even if the technical case looks strong.

Tools like FastQC and bwa mem have no formal publications associated with them. They are still widely used and are accepted to be the "(one of) best" for their respective applications.

• 0 views • link
written by GenoMax GenoMax

While this is well written, as someone who worked in a similar situation in an academic environment, in a core unit and in industry, I strongly disagree that great tools need to be published to be useful or adopted. Also my PIs would never have cared at all if a tool was published.

Coming from a HPC use case, from a sustainabilty or practical point of view, I think rewrites of old, slow, poorly maintained tools in a fast language like Rust are an absolute godsend for the community. There is too much poor and slow software out there with way too many complex dependencies.

• 0 views • link
written by colindaven colindaven

maybe it would make sense to plug it in polars-bio ? I'd happy to help with that!

• 0 views • link
written by Marek • Marek

Oh nice! Honestly, because I'm going for 1:1 emulation of the upstream tools only, I'm not quite sure what that'd look like. But it's nice to know it exists!

• 0 views • link
written by Phil Ewels Phil Ewels

I will take a look at RustQC and try to reuse it - I'm also prototyping with adding support for QC in polars-bio - maybe I can reuse some of the code if possible.

• 0 views • link
written by Marek • Marek

Log in to answer this question.

More posts like this