This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Creating a sample specific directory on the go while using -wget command to download the files

I want to download fastq files (> 1000) using wget . But while donwloading I want to save it to a specific directory that matches the sample name. For instance, if I have paired end reads, SRR22859377_1 and SRR22859377_2 , I want the directory to be named as SRR22859377. However, I want this to automatically happen while downloading for every sample, without creating the directory before.

I am having more than 1000 samples and don't want to create a directory everytime I download a sample.

Is it possible?

Thanks! :)

wget ncbi
• 2,390 views • link
updated by Ram Ram • written by Hrituraj • Hrituraj

if possible use wget --force-directories , otherwise just write a shell script...

• 0 views • link
written by Pierre Lindenbaum Pierre Lindenbaum

If the above does not work for you, you can give a try with the following, I have not tested the code though.

Assuming you have file with the accessions of interest, you can make use of wget's -P option to download to a directory of your choice. wget would create this directory for you if it does not exist.

#! /usr/bin/env bash

base_url="..."

while read accession; do
  wget "${base_url}${accession}_1" -P "${accession}"
  wget "${base_url}${accession}_2" -P "${accession}" 
done <accessions.tx
• 0 views • link
written by Haci Haci

What is accessions.tx here??

• 0 views • link
written by Hrituraj • Hrituraj

A file with accession numbers you need to get. One per line.

• 0 views • link
written by GenoMax GenoMax

It shows accessions.tx is no such file or directory.

• 0 views • link
written by Hrituraj • Hrituraj

You need to make that file.

• 0 views • link
written by GenoMax GenoMax

The base_url having issues. Could you give a few details how the base_url should be, because it connects to the ftp, but fails to download the file.

• 0 views • link
written by Hrituraj • Hrituraj

base_url will be the part that each accession's ftp/http link has in common.

• 0 views • link
written by Haci Haci

Well not exactly the directory I want using -- force-directories, it creates something like ftp.sra.ebi.ac.uk/vol1/fastq/SRR228/068/SRR22851168

This isn't the way I want. o_0

• 0 views • link
written by Hrituraj • Hrituraj

Please do not use bioinformatics as a tag unless your post is about the field of bioinformatics itself. For proper examples, please see Forum and News type posts under https://www.biostars.org/tag/bioinformatics/

I've removed the tag this time but please be more mindful in the future.

• 0 views • link
written by Ram Ram

0 answers

No answers yet.

Log in to answer this question.

More posts like this