This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Fastq-Dump Crashes, how to prevent besides continually restarting?

Hello,

I'm downloading public data files from NCBI using (for example) 

fastq-dump --split-files -F --bzip2 SRR2637659 > SRR2637659.out 2> SRR2637659.err

however, these commands crash every 6 runs or so, and download very very slowly. So far the only solution is to simply start again when fastq-dump fails.

The problem is that after I leave work I cannot restart this, or when I'm sleeping. Is there any option or clever trick so I don't have to keep manually re-starting this every time fastq-dump fails?

or even better yet, is there an option that for a given accession number, e.g. GSE88888, I can download all fastq associated with this data set? that would simply downloading greatly as well.

thanks

sra fastq-dump ncbi
• 2,363 views • link
written by dec986 dec986

What's the error it's failing with? For downloading all at once, look at this answer.

Edit: Got beat to it, his solution is easier.

• 0 views • link
written by jared.andrews07 jared.andrews07

1 answer

is there an option that for a given accession number, e.g. GSE88888, I can download all fastq associated with this data set?

Use EBI-ENA to directly download fastq files for your accessions avoiding these issues. As an example your accession # above.

• 0 views • link
written by GenoMax GenoMax

unfortunately, EBI-ENA has different issues. The "Bulk Download Files" button produces an error on Firefox "The address wasn’t understood

Firefox doesn’t know how to open this address, because one of the following protocols (jnlp) isn’t associated with any program or is not allowed in this context.

You might need to install other software to open this address."

I tried this same page on Safari and it tells me to do this on Firefox :/

However, I solved this: I should note that their downloading help page isn't so straightforward https://www.ebi.ac.uk/ena/browse/read-download when you need a * in order for wget to work (Silly me)

thank you @genomax!

• 0 views • link
written by dec986 dec986

You must have some local restriction on Java/jnlp code or don't have Java installed. When it works bulk download is convenient. For future reference: You can also click on the TEXT link to get a spreadsheet that has the actual FTP links that can be parsed when you have multiple samples in a bioproject.

• 0 views • link
written by GenoMax GenoMax

Log in to answer this question.

More posts like this