@Michael Hi, I did read your comment and used this same exact command line but the echo returns the exact same result as above.
:/mirza/Downloads/softwares/bowtie2-2.2.9/index:/home/mirza/Downloads/softwares/tophat-2.1.1:/mirza/Downloads/softwares/bowtie2-2.2.9/index:/home/mirza/Downloads/softwares/bowtie2-2.2.9/index:/home/mirza/Downloads/softwares/tophat-2.1.1
Still unable to run tophat from any other dir (returns the error: couldn't find b2 index files)
- should I be using this exact command line or some modification is needed? My tools are installed in /usr/bin (as i hv mentioned above).
- Do I need to undo adding the path I hv added earlier (/home/mirza/Downloads.......) and if yes, how to do it?
Very stupid check question:
are you doing
echo $PATHfrom the same terminal, or from another one?Well, thanks for the language. You might be an expert in linux but I am really new to it and am trying to learn so that I can run these tools for my research work. Anyways, I did realize my mistake that I was checking in the same terminal.
Considering, you are an expert and I am stupid, may be you can also shed some light on this, when I am trying to run tophat from a dir containing the files to be mapped, it returns the error, Error: Could not find Bowtie 2 index files (I already built the b2 index, contained in the b2 index dir)
Macspider wasn't calling you stupid, he/she was calling his/her own question "stupid".
Anyway, what are you using for the
<bowtie_index>argument and where are the indices?Chill :D the stupid thing was my question, but every question is worth asking!
If you export a variable from terminal 1, and then try to call it from terminal 2, you might not retrieve it (they load environmental variables when you open them).
If you want to have the command in your path, best thing is to edit your .bash_profile (or .profile, or .bashrc, depending on where your $PATH variable is declared) adding:
at the end of it.
For the bowtie2 indexes, you have to specify the --basename. This means that if your indexes are all called like whatever.bt2 you have to specify
-x whateverin the bowtie2 command. ;)I think $home should really be /home. export PATH=$PATH:/home/gjjha/Downloads/softwares/bowtie2-2.2.9/index:/home/gjjha/Downloads/softwares/tophat-2.1.1 Having said that, it would be better not to put random folders in the PATH, as Michael mentioned.
You should also be able to install bowtie2 (2.2.6-2) and tophat (2.1.0) from the Ubuntu Software Center.
Hi everyone. @Macspider sorry, I misunderstood, my bad. Couldn't clarify yesterday since I am a new user and can post only 5 times. I am writing in details so that I don’t miss any point this time. I am using ubuntu 16.04 on my workstn (256gb RAM). I have downloaded and installed bowtie2.2.9 and tophat2 according to the instruction given on tophat page of ccb.jhu.edu. I am successful in building indices and running tophat but only from the bowtie2 index directory, where the b2 index files are. This limits my usage coz I need to copy my raw read files in the index directory every time and can run one set of reads at a time only. So, I tried adding the location of these tools and the index dir to the path thinking it will enable me to run multiple files parallelly, from any other directory. Now I understand that these tools are installed in /usr/bin as “which” returns /usr/bin
$ which bowtie2 returns /usr/bin/bowtie2
$ which tophat2 returns /usr/bin/tophat2
But yesterday, using answers in some post, I used this command for adding the location of index directory to my PATH
$ export PATH=$PATH:$home/mirza/Downloads/softwares/bowtie2-2.2.9/index:/home/mirza/Downloads/softwares/tophat-2.1.1
echo $PATH returns
:/gjjha/Downloads/softwares/bowtie2-2.2.9/index:/home/gjjha/Downloads/softwares/tophat-2.1.1:/gjjha/Downloads/softwares/bowtie2-2.2.9/index:/home/gjjha/gjjha/Downloads/softwares/bowtie2-2.2.9/index:/home/gjjha/Downloads/softwares/tophat-2.1.1
I am still unable to run tophat2 from any directory other than the index (containing the indices) dir within bowtie2 dir. I need help with,
How to add these tools to my path?
How to undo the above command I used to add the index dir & tools to my path?
Did you read my answer?
@Macspider thanks I'll try these options and let you know tomorrow. @Michael Dondrup I have answered your reply below.
So this means that if you use the full path of the read files you don't get them? If you have these files: R1 = /path/to/reads_1.fq R2 = /path/to/reads_2.fq INDEX = /path/to/index/basename (don't consider the *.ht2 extension)
You should be able to run tophat2 from everywhere specifying full paths. Like:
No need to copy files!
Moreover, if you need to run many sets, you can either run many tophat runs changing the input files or using their special list comprehension in the input, separating by a comma the files you want to input (comma, without space afterwards), like they write in the program's help:
@Macspider One question, when we run tophat for a file, the results are saved in the tophat_out directory. So, when we are using the command tophat [options] <bowtie_index> <reads1[,reads2,...]> [reads1[,reads2,...]]<="" p="">
to run multiple PE sets, do we need to specify an output dir for each pair too or it will create separate directories itself?
Yes you do need to specify a unique new output directory for each sample. Every sample needs to be run independently. This is important since output from tophat has files with exactly the same name(s) for every sample.
There is a difference between running 2 tophat runs on two sets, and 1 run on two sets. In detail:
@Macspider I hv paired reads for replicates and different experimental conditions. I definitely want separate results not merged. Now how can I run them in parallel, to save time & efforts and how can I define separate output dir for each pair in one command?
Tophat has a -p option that sets how many threads you want to use. This has an upper limit in how many cores your machine / cluster has available, or in how many cores you are allowed to use if you are using some sort of queue manager.
That said, if you want to do parallel runs there is no built-in pipeline to do that, you have to do it yourself, but it's just as easy as launch N times the same command changing the files.
Since you don't want to overheat your system or to force more threads than the tolerated number in your cores, you'd probably be safe if you use for each run a number of threads (-p option) that is the result of:
Number of cores you can use (f.e. 30) / Number of runs (let's say 6) = 5 threads (in this example)
The power tool for running things in parallel is gnu-parallel, which can also restrict memory usage, specify the number of jobs, avoid swapping, show progress...
Or alternatively more complicated pipeline tools such as snakemake
@Macspider
@Devon Ryan
Hi, can you please tell me how to use the -output-dir option in tophat I have tried all these
-o my_out;
--output-dir my_out;
-o ./my_out
to creat my tophat_out directory but still no success. It creates its default dir tophat_out everytime.
I want to thank you all, learning a lot through biostar.
You should just specify a word and include an eventual path, it will create the folder. Can you paste here the command?
Hi, I am simply using
tophat b2.index input_file1 input_file2
where 1 & 2 are the left and right reads and its working fine. The output is generated in the default tophat_out directory.
But how did you use the -o flag?
as I have written in my earlier reply above, I tried using it in 3 different ways but still got the default dir not mine
tophat b2index input1 input2 -o my_out also,
tophat b2index input1 input2 --output-dir my_out and
tophat b2index input1 input2 -o ./my_out
Sometimes order matters, try
tophat -o my_out b2index input1 input2.ok, may be that's the problem. I can try this tomorrow only. Will post the outcome here. Thanks.
Indeed that's likely the problem. From the manual:
-o is an option and should come before the index. That works for me.
Just a side note to this:
You were lucky that this didn't mess up your data! The positional arguments at the end of the tophat command are considered as the input data, and the ouptut directory is one optional argument (--output-dir). If the output was as well a positional argument, you could have attempted to overwrite your input data with a null output file just by placing it in the wrong slot of the command line.
Bottom line: always be sure of what the manuals and helps say about arguments, if they're to be placed here or there, because this can really mess up things sometimes :)