This is a test version of Biostars. For the public version, visit https://www.biostars.org.
fastq-dump error for restricted data from dbGAP

I have got approval and downloaded sra files of restricted dataset from dbGAP. However, I am getting the following error while dumping to fastq. Here are the steps I used.

Step 01: download sra (successful)

prefetch --ascp-path  '/fullPath/ascp2.7/asperaweb_id_dsa.putty' --max-size 100G --ascp-options "-T -k 2 -l 400M" SRR3819129

Step 02: dump fastq files (error)

/fullPath/sratoolkit.2.8.2-1-ubuntu64/bin/fastq-dump --split-files --gzip /fullPath/phs000435/SRR3819129.sra

Error message:

2018-07-24T03:41:19 fastq-dump.2.8.2 err: item not found while constructing within virtual database module - the path '/fullPath/phs000435/SRR3819129.sra' cannot be opened as database or table

However fastq-dump works successfully for the public data SRR390728 using the same procedures. 

fastq-dump -X 5 -Z SRR390728

Additionally, I tried using "vdb-config --restore-defaults" but did not work. It would be great if you can help me with that.

Thanks in advance!

sequencing next-gen
• 4,491 views • link
updated by finswimmer finswimmer • written by Rashedul Islam Rashedul Islam

2 answers

I finally figured out the issue after email communications with sra team. I only had the download permission and therefore I was not able to extract raw sequences. I got the NGC file with full permission from my PI. The instruction was "The dbGaP repository key is only available from the dbGaP account of primary PI of approved data access request. It is not available from downloader’s dbGaP account. Downloaders should get the key directly from the PI."

Although the error message I got did not mention that I dont have proper permission until I came to know from sra team. Here is the summary of how I downloaded dbGAP restricted data.

1) From project directory, configure sra-tool $vdb-config --import prj_xxxx.ngc ./.
2) Downloaded sra files using ASPERA.
3) Test sra file: $test-sra SRRxxxxxxx.sra
4) Dump fastq files: $fastq-dump --split-files --gzip ./sra/SRRxxxxxxx.sra

• 0 views • link
written by Rashedul Islam Rashedul Islam

You have to execute the fastq-dump command from within /fullPath/phs000435/, so the same directory that you specified in vdb-config as the directory to load the restricted data into.

• 0 views • link
written by ATpoint ATpoint

Thanks a lot for your quick reply. Still have the problem. Steps I used:
-goto the wd
$cd /fullPath/phs000435/
-configure vdb
$vdb-config --import /fullPath/prj_1xxx1_D20xxx.ngc /fullPath/phs000435/
-download sra
$prefetch --ascp-path '/fullPath/ascp2.7/asperaweb_id_dsa.putty' --max-size 100G --ascp-options "-T -k 2 -l 400M" SRRxxx2334 -dump
$/fullPath/sratoolkit.2.8.2-1-ubuntu64/bin/fastq-dump --split-files --gzip /fullPath/phs000435/sra/SRRxxx2334

I ran fastq-dump from the directory specified in vdb-config e.g., /fullPath/phs000435/

• 0 views • link
written by Rashedul Islam Rashedul Islam

Hmm, I just tried to re-download a restricted file that we have access to and I successfully loaded and dumped last year, but it always prompts that access was denied. Maybe a problem with dbgap itself. In that case I would contact their support.

• 0 views • link
written by ATpoint ATpoint

Thanks a lot for your help. I also check other discussions on that issue. I emailed sra developer.

Update: 1. made ncbi root directory to my project directory. 2. imported config file to project directory 3. downloaded sra file to project directory 4. and getting same error in fastq-dump while executing it from the project directory.

When I ran test-sra SRR3652334.sra and test-sra from the project directory. In both case its showing total space is 0 and free space is negative, although I have 8TB of space in my project directory. Can you check test-sra from your directory?

<Space free="-442525664" total="0" units="KBytes"/>

• 0 views • link
written by Rashedul Islam Rashedul Islam

Log in to answer this question.

More posts like this