Hello!
I am studying RNA modifications, and I want to use nanopolish to then detect them with tools like m6anet.
I downloaded fastq files from a study using the SRA toolkit, and then downloaded the raw fast5 files from the links available in SRA page, located in the AWS cloud. Then, I ran nanopolish index, and I got the warning (I censored the path):
[readdb] indexing /path/to/my/fast5
[readdb] num reads: 258957, num reads with path to fast5: 129495 fast5 files could not be located for 129462 reads
So apparently, the fastq has more reads than the fast5 files, but the indices were done anyway. The total reads declared both in the SRA sample page and the European Nucleotide Archive (ENA) are 129,462, and nanopolish says 258,957.
Running grep -c '@' SRR31451028_1.fastq gives me 254181 occurrences (I know it is not the best approach to count the reads). Anyway, it seems that the fastq indeed has more reads than fast5. I also downloaded the fastq through the ftp link in ENA, and checked if it was corrupted by comparing the md5sum of my local copy with the one on the ENA website. Everything was fine, same md5sum, and still, I got the same message when running nanopolish index. Even the number of total reads and the number of missing reads are the same.
I have no clue what could be happening here. I am still kinda new in this nanopore world. Does someone have any idea? Is it normal, and can I just continue with the analysis?
Thank you!
0 answers
No answers yet.
Log in to answer this question.
-
How do I use md5sum to check if my .fastq files from SRA Explorer have been downloaded correctly
written by biotrekker •
-
SRR download using fasterq-dump
written by Biomed-jeh •
-
Convert Nanopore Fast5 files to Fasta format
written by Sowmya Pulapet •
-
Processing Fast5 files from ONT for direct RNA
written by Shagnik •
-
Three fastq files in ENA for paired sequencing - PRJEB3381?
written by Dhana
-
ENA and SRA fastq files
written by Sbrillo •
-
What is the recommended way to subsample Oxford Nanopore (ONT) fastq reads with matching fast5?
written by O.rka
-
Minion RNA-seq local basecalled .fast5 dataset
written by adeline •
-
How can I change the default path to which SRA files are downloaded?
written by Cacau
-
Sra-Toolkit 2.3.1 Fastq-Dump Multiplefiles At Once Windows Command Line
written by deepbiofever •
Have you looked at
modkit? It is the current method to work with modification data https://github.com/nanoporetech/modkitNo, I actually never heard of it before! I will take a look, thank you. However, this study is old; it used the SQK-RNA002 kit. Would you happen to know if this tool is also for these older sequencings?
will potentially count much more than only the sequence header lines (it can occur as well in the score line)
changing it to
grep -c '^@'will reduce most of the 'false' positives (but not all obviously ;) )more on topic: perhaps read-splitting (during the basecalling process) can account for part of the discrepancy? (== 1 entry in the fast5 can result in multiple in the fastq file)
Thank you! Using
grep -c '^@'helped me reduce a lot the FP!However, I did check now, and even though I used the --split-files option, I have only 1 fastq file per sample. Now I also tried downloading the data with --concatenate-reads, and now it says that I downloaded 129,462 reads, as expected (reads written: 129,462). However, the nanopolish index still gives me the same error :( Moreover, I tried nanopolish eventalign anyway, but it returns an empty file with no errors returned.
If I am not mistaken;
--split-filesis only used to split up short-read paired-end reads into between 2-3 files containing pair1, pair2 and unpaired reads (if present). So, it makes sense that the output contains only a single file.Not sure if it is helpful but It is curious that you have almost exactly twice the number of fastq reads as you do fast5 (only a difference of 33 reads). I wonder if any of the fastq files have the same name and are duplicated? You could quickly check with something like
grep -P "^@" *.fastq | sort -u | wc -lHello! I used
grep -P "^@" *.fastq | sort -u | wc -lin the folder with all the samples' fastq, and the total it returns is the same as what I expected for the total of reads of all samples, as stated in ENA and SRA. I also used it only in the sample I used here as an example, and it returns 129463 lines. Apparently everything's fine with the fastqs... I was wondering if it could be that the authors uploaded only the "pass" fast5 files, but the fastq were produced with all of the files, including the failed ones.You could check the read quality of "extra" reads in fastq and see if they have poor quality.
It is possible; however, I think it is unlikely that 50% of the reads failed (at least with the default filter) So, yea, as below, you could check the quality of the remaining reads as suggested by GenoMax
The check above is just if the fastq headers are the same. You could check to see if the reads are the same using seqkit, regardless of headers, then check the number of fastq reads again; should be something like
seqkit rmdup -s *.fastq | grep -P "^@" | sort -u | wc -l(pretty sure you can just pipe from seqkit; the-soption remove duplicates based on identical sequences)Thank you for all the answers! I checked in all the possible ways stated here in the comments, including with seqkit, to try to remove duplicates, if any. Apparently,y the fastq is fine, and it indeed has the number of reads it should have. It seems that this error also occurred to another user with nanopolished, so it could be something there and not the files per se. However, trying to create the index with f5c (which is also compatible with Nanopolish) results in an error, as it cannot locate the fast5 files
So I keep investigating this issue