This is a test version of Biostars. For the public version, visit https://www.biostars.org.
How to download protozoa reference genomes from NCBI

I want to download all available protozoa genomes from the NCBI database. Using ncbi-datasets download is unfortunately not an option, as it doesn't recognize "protozoa" as a valid taxon.

However, I found that the genomes are shared via FTP at the following locations: GenBank - protozoa RefSeq - protozoa

I tried downloading the contents using wget, but it only retrieves the directory listings — the actual genome files inside the subfolders (e.g., .fna.gz, .gbff.gz, etc.) are not being downloaded recursively.

This is the command I used: wget -r --continue --progress=bar:force:noscroll ftp://ftp.ncbi.nlm.nih.gov/genomes/genbank/protozoa/

How can I modify this or use another method to properly download all genome files, including those in the subdirectories?

Any help or suggestions would be greatly appreciated!

P.S I'm also interested if there is any way to download only microfungi genomes.

genomes ncbi
• 1,332 views • link
updated by colindaven colindaven • written by anna • anna

2 answers

I found a solution of my problem. In case it will be helpful to anyone I leave a workflow here:

  1. Download assembly_summary.txt file

    wget ftp://ftp.ncbi.nlm.nih.gov/genomes/genbank/protozoa/assembly_summary.txt

  1. Extract FTP paths matching required parameters

    grep -v "^#" assembly_summary.txt | \ awk -F '\t' '

    $25 == "protozoa" && $11 == "latest" && $5 == "reference genome" {
print $20 > "protozoa_ftp_paths.txt";
print $0 > "protozoa_assembly_summary_filtered.txt"
   }
  1. Append the filename based on the directory name.
 mkdir -p protozoa_fna

    cd protozoa_fna

    while read dir; do
        base=$(basename "$dir")
        file="${base}_genomic.fna.gz"
        full_url="${dir}/${file}"
        echo "Downloading $full_url"
        wget -c "$full_url" --progress=bar:force:noscroll
    done < ../protozoa_ftp_paths.txt
• 0 views • link
written by anna • anna

I know they're not the same, but you might be interested in https://protists.ensembl.org/index.html as well. I don't know how up to date and consistent it is compared with NCBI.

• 0 views • link
written by colindaven colindaven

Log in to answer this question.

More posts like this