Hello all,
I'm beginning to learn Python 2.7 and need to develop some applications for a lab project.
I want to run an external program and then use its output as input for another external call in the same script. The issue is that the script tries to call the second program even before the first one has finished working, so the output is not yet available to be used as its input. How can I solve this problem?
Here is my code:
picker = Popen ([pick_open_reference_otus.py, -i, allseq_file, -r, refseq_file, -o, noName, -s, 0.1, -m, usearch61, -p, param_file ], shell = True)
picker_out = os.path.join(path, "otu_output/otu_table_mc2_w_tax.biom")
Thank you all!
•
written
by
uxue33 •
1 answer
I used the following code for similar jobs:
retmode = subprocess.call(shlex.split('your shell command with arguments'))
if retmode > 0:
sys.exit("Prinseq returned error code {} for sample {}".format(retmode, target))
I find the shlex module useful to structure command prior to using subprocess.call, it allows you to just write the string of the command and it will take care of the formatting in a list.
Dear all,
I found this post very useful and tried to use the solution myself. The thing is that I presume I get an error in the variable interpolation process that I can't figure out how to fix (I haven't used Python very often for this kind of task, because I usually use Perl for external program related pipelines). Here's my code and the error it produces.
import sys, os
import subprocess
import shlex
allseq_file = "/home/jllavin/Metagenomics/Test.fasta"
refseq_file = "/home/jllavin/Metagenomics/Dbs/99_otus.fasta"
picker = subprocess.call(shlex.split('pick_open_reference_otus.py -i allseq_file -r refseq_file -o output2 -s "0.1" -m "usearch61" -p parameters.txt -f'))
if picker > 0:
sys.exit[(picker)]
The error:
Traceback (most recent call last):
File "/usr/lib64/python2.7/lib-tk/Tkinter.py", line 1470, in __call__
return self.func(*args)
File "QIIME_shlex.py", line 45, in button_ex_callback
sys.exit[(picker)]
TypeError: 'builtin_function_or_method' object has no attribute '__getitem__'
hi joluito,
for sure sys.exit[(picker)] is wrong. The square brackets don't serve a purpose here. Try sys.exit(picker).
(This is just the return code of the script, in case something goes wrong)
for the rest, I don't think your -i allseq_file is going to work. Or is allseq_file an actual file in the current directory?
If it's a variable in your python code (what I suspect) you need to construct the string differently:
picker = subprocess.call(shlex.split('pick_open_reference_otus.py -i ' + allseq_file + ' -r ' + refseq_file + etc etc'))
As such the value of the variable is inserted in the string, fed to shlex.split() and used for the process.
Note the spaces!
It can be helpful to first construct your command:
cmd = 'pick_open_reference_otus.py -i ' + allseq_file + ' -r ' + refseq_file + etc etc'
print(cmd)
picker = subprocess.call(shlex.split(cmd))
and print that to see if everything is okay
So as I suspected, you are passing the variable's name to pick_open_reference_otus.py, rather than the variable's value. Is that clear?
Interesting point you raised here.
So, I'm not interpolating the variable's "string value" by passing its "name" to the command...
Therefore the way to proceed is to generate the command in a "print style" and then using it via subprocess.call(shlex.split)
Thank you very much for your patience, I will try it and come back with the outcome.
Best
Yes, just concat the strings together with the variables and don't forget spaces.
Log in to answer this question.
-
My kernel is killing the pairtools dedup script, how to prevent that?
written by NikhilP •
-
snakemake how to get or create wildcards from newly generated files
written by gb
-
PoolSNP RuntimeError: dictionary changed size during iteration
written by shpak.max •
-
Passing multiple arguments to external programs in a Pipeline with Python
written by barslmn •
-
Python 2.7 Tk filedialog calling external programs
written by uxue33 •
-
Execute external program in Tk Python 2.7
written by uxue33 •
-
Speed up PHASE software for haplotype inference
written by kshitijtayal •
-
Orthology detection using proteinortho
written by Ritvik •
-
Can I input sequences without a file clustalw2
written by alex •
-
To Call A Perl Script As An External Program In A Php Script
written by grosy •
I'm not 100% sure because I have not done any work with the subprocesses module, however I think you can just use the wait() module. something like
picker.wait(picker_out)or something to that effect.or
picker_out = picker.wait(os.path.join(path, "otu_output/otu_table_mc2_w_tax.biom")sorry. I'm not sure about the syntaxlast one :)
Thank you a lot for your help!
https://docs.python.org/2/library/subprocess.html
Subprocess has what you want.
Python can be rather clunky for this kind of thing. If you need data from some of these programs you're calling to end up being processed in python, it is better to pull the data into python from the "outside". You may want to explore other options for managing pipelines, there are plenty of posts and tutorials on here about it.
Thank you for the link!