I am having issues downloading single cell RNA-seq fastq files from SRA Explorer. Files are corrupted and sometimes half the size. I have 200+ files so it becomes hard to check which ones need to be redownloaded. How can I check using md5sum that the files I downloaded are correct. I am using a virtual machine to download this massive amount of data
Thank you
•
written
by
biotrekker •
As far as download goes, I don't think that sra-explorer has any option for md5sums. Here is two ad-hoc options:
1) Use touch: wget (options...) <file> && touch file.done
That will only create an empty file called file.done (or any name you give it in your script/loop) if download finished and wget did not throw an error. I think if connection breaks then this should not be created. So presence of the file indicates proper download.
2) Use Aspera, which sra-explorer provides download links for.
The tool is clever enough to download data as a tmp/hidden file and only create the final visible file if download completed successfully. Presence of final file means download was ok. See Setting up Aspera Connect (ascp) on Linux and macOS
Say I have 300 samples I am downloading from sra-explorer, do I add "&& touch file.done" at the end of each of the 300 curl commands?
Additionally I am having trouble setting ASCP virtually, is there a different way to set up aspera connect virtually?
Thanks
Log in to answer this question.
You may need to use
sratoolkit prefetchand then check usingvdb-validatebefore dumping the data out. Ideally you would get the original BAM files (if submitted by submitters) and then use the 10x util (bamtofastq) locally.Single cell data is all over the place in SRA and unfortunately
sra-explorerdoes not help with that.Hi, may be this could help you :
https://github.com/ncbi/sra-tools/issues/531
Best
fyi, the SRA-explorer is a third-party tool, not one from NCBI.