I'm trying to blastx a Fasta file with a little over 23000 genes in it. I have downloaded the full nr database and unzipped all the tar.gz files. I have downloaded the standalone Blast tool. This is the command I am running:
blastx -query "C:\Program Files\NCBI\blast-2.17.0+\bin\Ha_Trinity_denovo_assembly.fasta" -db nr -outfmt 10 -out "Ha_blastx_results.csv" -max_target_seqs 100 -num_threads 4
I am using the correct path for the Fasta file. I am running it in a command line from the folder that contains the database. But after waiting for 2 hours, nothing has appeared in the .csv file. There is no error message in the command prompt window. It just shows that the command is still running. I also tried this with one gene (362bp) and waiting 10 minutes, and one gene with the output file being a text file instead of csv. The problem is the same every time. Any idea what I might be doing wrong?
Sorry if this is a very base-level question, I have looked everywhere online and can't find an answer.
•
written
by
pahurkar.arjun •
2 answers
Any idea what I might be doing wrong?
Chances are you are not doing anything wrong. You simply have inadequate resources. As this seems to work with a single, short sequence, I would guess that with longer sequences and with your resources it will take a very long time. I don't want to be gloomy, but it is very unlikely that you can do this project on your computer in a finite period of time. It would probably take days to weeks to search 23000 sequences even if you had 512 GB RAM and 50+ cores. I can't even predict how long it would take with your resources, but I think years.
I think you will have to split your sequences into several groups and run them simultaneously on a computer cluster with much higher resources. Better yet, translate your ORFs into proteins and that will speed up things at least to some degree.
If I used 3 virtual machines and split the fasta file between them, and translated the full file from nucleotides to proteins before starting, how much time do you think it would take?
how much time do you think it would take?
You will need the same resources, RAM and cores, for all three machines. We don't know the length of sequences in your dataset and/or config of VM you will use. You will need to experiment and try this yourself with a smaller subset to get an idea.
That said what is your aim here? Based on the file name it sounds like this is some sort of assembled transcriptome. Do you need to blast against entire nr? You could use a smaller set of sequences and/or close relative to do this and things should go much quicker.
Also consider using DIAMOND instead of blast+, which is more efficient for this sort of thing. You should expect similar compute needs and since you appear to have access to cloud compute this would be a viable option. DIAMOND can use pre-formatted blast indexes so you can re-use those.
If your goal is to annotate the genome, BLAST is inefficient both in terms of time and the ability to do it well. Most modern approaches for this problem include databases of profiles or hidden Markov models (HMMs). For example, RPS-BLAST can annotate 23,000 proteins against the NCBI profile database in a matter of hours, and it is almost guaranteed to produce better results than a brute-force BLAST searching. Similarly, you can download an HMM database from InterPro and annotate all your proteins with HMMer in a matter of hours. I suggest you think hard whether BLASTing this many proteins is actually needed to achieve your final goal before you invest more effort.
Log in to answer this question.
-
Issues while running blastx
written by Foad •
-
BLAST error: Input db vol size does not match lmdb vol size
written by Hansen_869 •
-
BLAST 2.11 hanging when using local nr database
written by scarey •
-
BlastX Database: output search is empty
written by marongiu.luigi
-
BlastX Database error: No alias or index file found for protein database
written by marongiu.luigi
-
local blastp against NR database
written by Chirag Parsania
-
BLAST options error: tried to set option (130) and value (1), line (537)
written by oliver.bayfield •
-
Blast Error: Tax database file not found
written by navillusol858 •
-
For Loop Is Not Working Within My Script
written by federico.gaiti
-
Help For Blastx, No Output Untill The Program Finished
written by zhangmin8712 •
How much memory does your windows machine have? With
nrdatabase you will need at least 50-60 GB of RAM.Hi, thanks for the reply. I currently have the nr database on a hard drive which has about 450 GB remaining. My computer says I have 16 GB of installed RAM. The local disk has about 160 GB remaining. I'm not great with computers, so I don't know if you're asking about my installed RAM or just memory, sorry about that.
Also just some additional information that I forgot to include in the original post, my computer has 4 cores, and I'm using all of them for this.
make a out file using YOUR-COMMAND > C:\PATH\TO\FOLDER\OUTPUT.txt and you can see that something is going on or not
This question has been addressed and has answers. Your command would also not help in figuring out if there's progress. When there's no output to STDOUT or STDERR, why would writing STDOUT to a file be of any use?