This is a test version of Biostars. For the public version, visit https://www.biostars.org.
A question of NGS requirement

Hi

Please, could somebody give me a hand about some infrustructure information?

storage rnaseq
• 4,356 views • link
written by zizigolu zizigolu

3 answers

It surprisingly may not be that large in terms of size for prokaryotic samples.

Here is a table of storage size estimates from GPT for different types of prokaryptic transcriptomes for one sample (Typically 50–150 bp paired-end (e.g., 2 × 75 bp or 2 × 100 bp).

| Data type                             | Approx. reads        | Approx. file size (FASTQ, gzip-compressed) | Notes                                                                      |
| ------------------------------------- | -------------------- | ------------------------------------------ | -------------------------------------------------------------------------- |
| **Basic differential expression**     | 5–10 million reads   | ~0.5–2 GB                                  | Sufficient for most bacteria with small genomes (~3–6 Mb).                 |
| **High-depth transcriptome**          | 20–50 million reads  | ~2–10 GB                                   | Used for lowly expressed genes, isoform detection, or complex communities. |
| **Metatranscriptome / mixed culture** | 50–150 million reads | ~10–30 GB                                  | Needed to capture transcripts from many species.

You can multiply that by the number of samples. You can double the storage to account for all downstream analysis files to manage everything comfortably.

• 0 views • link
written by GenoMax GenoMax

What kind of sequencing are you thinking of doing? Direct RNA or normal cDNA?

Assuming cDNA you may only need 10 cells (with barcoding and very deep sequencing) so roughly a total of ~ 2TB of sequence data. Less with lesser coverage.

| Sequencing depth per sample | 50 Gb flowcell (moderate) | 100 Gb flowcell (good) | 200 Gb flowcell (excellent) |
| --------------------------- | -------------- | --------------- | --------------- |
| 0.75 Gb (basic RNA-seq)     | ~65 samples    | ~130 samples    | ~260 samples    |
| 3 Gb (deep coverage)        | ~16 samples    | ~33 samples     | ~66 samples     |
| 7.5 Gb (very deep)          | ~6 samples     | ~13 samples     | ~26 samples     |

Direct RNA yields would be significantly smaller and barcoding support is lacking so you can only do apparently 4-5 samples per cell.

With polycistronic transctipts analysis may be trickier if you choose full length direct RNA sequencing.

• 0 views • link
written by GenoMax GenoMax

so If we also keep the raw signal files (FAST5/POD5) for possible re-basecalling later, the 60–70 TB over the full project makes sense or exaggerative?

• 0 views • link
written by zizigolu zizigolu

One can never have enough storage but that said the PromethION cells were more in the ~ 1.5 TB size in my experience. So the amount you mention should be significantly more than strictly needed.

• 0 views • link
written by GenoMax GenoMax

Thanks a million.

• 0 views • link
written by zizigolu zizigolu

You have not said if you are planning to sequence direct RNA or cDNA.

36 PromethION sounds like you intend to do direct RNA since that would be overkill for cDNA data for 250 samples. If you are sure about there being 36 PromethION flowcells total, then count about 1.5 TB for raw data per cell (as of now for cDNA, less for RNA). Projecting something 6 years out is difficult since the technology keeps moving rapidly, so adding 1.5-2x more to the estimate should be adequate for future improvements.

• 0 views • link
written by GenoMax GenoMax

Thanks a lot, very helpful comment.

• 0 views • link
written by zizigolu zizigolu

Sequencing could be done locally or by an external provider. If an external provider does the sequencing you will need to ask them to provide the entire data folder, if you intend to re-basecall, at later date.

Locally one could use a "P2 solo" accessory to run PromethION FC's (up to 2) with a beefy enough workstation (MinKNOW software works on windows as well as linux) or even a GridION sequencer.

While basecalling can be run on local computer during the run (assuming you have the right GPU and enough hardware), it may be simpler to allow the run to complete with FAST basecalling locally and then do high/super accuracy calling on external hardware (GPU(s)) using the raw data folder at any time later.

• 0 views • link
written by GenoMax GenoMax

Sorry, if I buy This is for Eureka2 GPU node 2 × NVIDIA L40S GPU 512 GB RAM 64 CPU cores 2-4 TB NVMe SSD Estimated cost: Approx £30k depending on the GPU chosen. do I need a workstation yet? Buying GPUs is preferrable or a workstation? Thanks a lot

• 0 views • link
written by zizigolu zizigolu

Eureka2 GPU node 2 × NVIDIA L40S GPU 512 GB RAM 64 CPU cores 2-4 TB NVMe SSD

I assume you are referring to a shared node on a compute cluster? If you are only going to run tens of flowcells over 6 years, then going with 1 GPU (if you are actually going to buy the shared node) may be plenty. If this is a node in the computer cluster then you are clearly not going to attach the P2 solo device to it directly during sequencing. So yes, you would need a reasonably good machine/workstation to collect the primary data elsewhere. You will then move the data folder to cluster for SUP/HAC basecalling as needed.

• 0 views • link
written by GenoMax GenoMax

Thanks for the helpful reply. Just to clarify the setup in our case:

The sequencing is not being done locally. All PromethION runs will be generated by an external sequencing provider, so we will receive the full raw data folders (FAST5/POD5 + metadata). We are not running MinKNOW or attaching a P2 Solo/PromethION device ourselves.

The compute question is only about the analysis side. We plan to do:

HAC/SUP re-basecalling of the provider-generated data

modified-base calling

and training a nanopore model (signal-level deep learning)

For this reason, I was referring specifically to a GPU node on our HPC (dual L40S, ~£30k), not to a local workstation for running sequencing.

Given this setup, I was trying to understand whether accessing a shared HPC GPU node would be sufficient, or whether an additional workstation with a single GPU would still be useful for model-development work.

• 0 views • link
written by zizigolu zizigolu

Shared GPU node should be adequate. Depending on the GPU speed/data size, it can take several hours to complete SUP/modified base calling for a promethION cell. So keep that in mind. It may mean waiting for sometime, if you are going to use a job scheduler and the nodes are busy.

If you have funds to buy a node, then it could become part of large cluster but still be made available on priority to your jobs. This way you don't have to administer the node and can use it as needed.

Note: It looks like you are removing parts of your posts/comments after they receive a reply. Please don't do that. While you are getting what you need others coming by this thread would be left puzzled by this exchange.

• 0 views • link
written by GenoMax GenoMax

Thanks a lot You are very right

• 0 views • link
written by zizigolu zizigolu

I just can say AI models I have tried could not give such a comprehensive answer. Thanks a lot once more

• 0 views • link
written by zizigolu zizigolu

Depends how much you do with them, how good you are at being disciplined with storage, what size of file are initially generated, whether you want to keep all the aligned bams, how many reference genomes you use, if you do de novo assembly too, etc etc etc.

Constructive points

  • pigz - parallel gzip - is your friend
  • program in nextflow pipelines and delete tmp files in the work dir all the fime
  • use a tool like ncdu or dust to monitor disk usage very frequently
  • be disciplined
  • check ncbi etc for the range of file sizes per sample you might generate. Multiply by 250.

Consider also offline storage as backup, eg amazon glacier, even multiple copies on local external hard disks can be very cheap.

• 0 views • link
written by colindaven colindaven

It depends on the sequencing depth, transcriptome size, and how many intermediate files are stored/created during analysis.

Example:

Mycobacterium tuberculosis; RNA-Seq (SRR7444071)

Genome size: 4.5Mb

Number of reads: 2,393,077

File size: 129.1MB

A conservative estimate will be ~700GB to 1 TB at max, assuming none of the files are stored in an uncompressed format.

• 0 views • link
written by Arup Ghosh Arup Ghosh

Log in to answer this question.

More posts like this