Batch Download from PubMed ftp server

Question

[pacman]

I have a list of articles (couple hundreds) that I would like to download the full text from PubMed central in PDF. Unfortunately, even though all of them can be found at PubMed central, I can only download less than 10% of them from the FTP service ,based on the file_list.pdf.txt. I am wondering if there is any other ways to do a batch download or just to speed up the process. Thanks!

[Tky]

This is a duplicated question.

How To Download A Few Hundred Pdfs From Pubmed

Answer 1

[Shicheng Guo]

I have tried. it works. I can download all the files or download whole directory from the FTP.

1. Download files

   wget ftp://ftp.ncbi.nlm.nih.gov/pub/pmc/08/e0/BCR-3-1-055.PMC13900.pdf ./
   

2. Download whole directory:

   wget -r ftp://ftp.ncbi.nlm.nih.gov/pub/pmc/08/e0 ./
   

[pacman]

Yes, it works, but not all PMC articles can be found on FTP server due to copyright issues :(

[Shicheng Guo]

Which one? Is there any example?

Answer 2

[pacman]

Hi,

I found a trick that make the process go a little bit faster.

1. Find the corresponding PMC ID (you can find it from PMC-ids.csv.gz)
2. Follow the format of this URL: http://www.ncbi.nlm.nih.gov/pmc/articles/PMCXXXXXX/pdf -> this URL will re-direct the browser to the download link if the pdf is available

I use an excel vba script to semi-automate the process by opening a browser and directing it to the generated URL. This process is not ideal but at least it will minimize the number of clicks you need to download the articles

Answer 3

[Danielson]

I'd say, the easiest way to download all the documents is using FileZilla. Host: ftp.ncbi.nlm.nih.gov /pub/pmc

There you can download them (those from PMC, ie the articles from your list (1.1 million)), in various formats. The root of that directory include a couple of tar.gz files; articles-* has a list of all the 1 million articles in nxml (xml format), the articles.txt.* in plain text. The latter seemed great, at first, but I found some issues with outlining.

Other files 'file_list*' list the journal issues including PMC_id, and the location where more information can be downloaded (a link to ftp.ncbi.nlm.nih.gov/pub/pmc/{hex}/{hex}/{article}. This includes pdfs, nxmls and images...