This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Downloading NCBI Blast nt database

I would like to download the full nt database to run a BLAST search on a large genome through HPC. I have installed BLAST using conda and set up accordingly.

When I run the step below (edited from the manual (https://www.ncbi.nlm.nih.gov/books/NBK569850/) for the nr database):

~/miniconda3/envs/blast/bin/update_blastdb.pl --passive --decompress nt

I get the error:

**Connected to NCBI
Downloading nt (90 volumes) ...  
Downloading nt.00.tar.gz...corrupt download, trying again.
Downloading nt.00.tar.gz...corrupt download, trying again.
too many failures, aborting download!**

Has anyone else encountered this issue? Thanks.

nucleotide blast nt genomics database
• 9,824 views • link
updated by size_t • size_t • written by rgray • rgray

If you have an intrusion prevention/detection device and/or a firewall between the server and internet then this could be causing the problem you see. You will need to make sure your connections are allowed through.

• 0 views • link
written by GenoMax GenoMax

Thanks for your reply. I thought the --passive option should get round this (--passive Use passive FTP, useful when behind a firewall or working in the cloud) but it still doesn't allow. Any advice would be great

• 0 views • link
written by rgray • rgray

See my comment below. If @Istvan's solution works for you then great.

• 0 views • link
written by GenoMax GenoMax

2 answers

You can always download the files from:

it is fairly simple to automate as well.

seq -w 00 89 | parallel wget https://ftp.ncbi.nlm.nih.gov/blast/db/nt.{}.tar.gz
• 0 views • link
written by Istvan Albert Istvan Albert

If a deep packet inspection device is causing this problem then the only way around it to get those network flows whitelisted.

• 0 views • link
written by GenoMax GenoMax

I have still had no luck, so suspect it could be a firewall issue with my hpc server. I am able to download files using wget, however when I try to extract the files (tar -zxvpf nt.89.tar.gz) to use them I get the error:

nt.nal
nt.89.nin
nt.89.nhr
nt.89.nsq

gzip: stdin: invalid compressed data--format violated
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

I will update what works if I am able to change the firewall permissions.

• 0 views • link
written by rgray • rgray

Thanks, I will give this a go too!

• 0 views • link
written by rgray • rgray

try this ：

ascp -T -l 200M -i ~/.aspera/connect/etc/asperaweb_id_dsa.openssh --host=ftp.ncbi.nih.gov --user=anonftp --mode=recv /blast/db/FASTA/nt.gz ./

ref：ascp ncbi

• 0 views • link
written by size_t • size_t

Log in to answer this question.

More posts like this