I'm trying to fetch fastq files for SRA data hosted by NCBI from AWS data exchange for a project, but the only way I can find is to use the SRA toolkit to convert the SRA normalized files in the sra-pub-run-odp S3 bucket. Is there a way to fetch these files directly as fastq files using any public APIs or data sources hosted freely to avoid the pain of conversion of files and bypass the use of the toolkit? Thanks!
2 answers
What is the accession for the project? Many — though not all — of the projects present in the NCBI SRA are also hosted in the European Nucleotide Archive (ENA), which directly provides gzipped FASTQ files, rather than the SRA format. As you mention, the SRA format is largely a pain and the most common use of it, by far, seems to be just as a way to slow down processing pipelines by first requiring conversion to FASTQ.
If you have the accession information, you can use something like SRA explorer to find the associated ENA files and obtain download scripts for them. Perhaps even more expedient, from the command line, you can use the fastq-dl tool to directly obtain the FASTQ files from ENA given the accession number.
You can also use Grabseqs.
Easiest way is to use it with anaconda as conda install -c louiejtaylor grabseqs
But you have to manually change 1 line in its code which is mentioned here
" Try replacing /usr/local/lib/python3.6/site-packages/grabseqslib/sra.py line 94 with
metadata = requests.get("https://trace.ncbi.nlm.nih.gov/Traces/sra-db-be/sra-db-be.cgi?rettype=runinfo&term="+pacc) "
Log in to answer this question.
Please provide an example of a SRA accession where you are encountering this.