I am trying to post query to a webserver : http://www.imtech.res.in/raghava/antibp/submit.html
but I am getting an error
Traceback (most recent call last):
File "crawler.py", line 4, in <module>
conn = httplib.HTTPConnection("http://www.imtech.res.in/raghava/antibp/submit.html")
File "/usr/lib/python2.7/httplib.py", line 704, in __init__
self._set_hostport(host, port)
File "/usr/lib/python2.7/httplib.py", line 732, in _set_hostport
raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
httplib.InvalidURL: nonnumeric port: '//www.imtech.res.in/raghava/antibp/submit.html'
The python script is shown below:
import httplib, urllib
params = urllib.urlencode({'seqname':'GICACRRRFCPNSERFSGYCRVNGARYVRCCSRR','format':'Amino acid sequence in single letter code', 'terminus':'N-terminus', 'method':'svm', 'svm_th':'0', 'type': 'Submit'})
headers = {"Content-type": "application/x-www-form-urlencoded", "Accept": "text/plain"}
conn = httplib.HTTPConnection("http://www.imtech.res.in/raghava/antibp/submit.html")
conn.request("POST", "", params, headers)
response = conn.getresponse()
print response.status, response.reason
data = response.read()
conn.close()
What could be the problem? Thank you.
3 answers
That's how I would do it, with the disclaimer that I'm no expert in querying web pages and I don't know anything about the server in question:
python
import mechanize
br = mechanize.Browser()
br.set_handle_robots(False)
br.open("http://www.imtech.res.in/raghava/antibp/submit.html")
br.select_form(nr = 0)
## See what is available on this web page:
for f in br.forms():
print f
#<POST http://www.imtech.res.in/cgibin/antibp/antibp1.pl multipart/form-data
# <TextControl(seqname=)>
# <TextareaControl(seq=)>
# <FileControl(file=<No files added>)>
# <SelectControl(format=[*nformat, sformat])>
# <RadioControl(terminus=[*1, 2, 3])>
# <RadioControl(method=[*1, 2, 3])>
# <TextControl(svm_th=0)>
# <TextControl(ann_th=0.6)>
# <TextControl(qm_th=-0.2)>
# <SubmitControl(<None>=Submit) (readonly)>
# <IgnoreControl(<None>=<None>)>>
## Input your sequence and parameters:
br['seqname']= 'myseq'
br['seq']= 'GICACRRRFCPNSERFSGYCRVNGARYVRCCSRR'
br['format']= ['nformat']
br['terminus']= ['1']
br['svm_th']= '0'
## Sumbit and collect results:
res= br.submit()
html= res.read()
Now html is string in html format that you could parse with an html parser or something else. The relevant bit in html should look like:
<td><font size="4"><b>Antibacterial Activiy</b></font></td></tr><tr>
<td align="CENTER">GICACRRRFCPNSER</td><td align="CENTER">1</td><td align="CENTER">1.975</td><td align="CENTER">YES</td></tr><tr>
<td align="CENTER">GYCRVNGARYVRCCS</td><td align="CENTER">18</td><td align="CENTER">1.051</td><td align="CENTER">YES</td></tr><tr>
<td align="CENTER">ICACRRRFCPNSERF</td><td align="CENTER">2</td><td align="CENTER">1.001</td><td align="CENTER">YES</td></tr><tr>
...
Don't use httplib (and other native libraries) directly. If you want to stay sane that is.
Have a look at the requests library instead, I bet you will be able to code your request just by reading the first page of documentation.
•
written
by
Saulius Lukauskas
This is really more of a python question than a bioinformatics one.
Only the server name should be included in HTTPConnection():
conn = httplib.HTTPConnection("www.imtech.res.in")
conn.request("POST", "/raghava/antibp/submit.html", params, headers)
I've not tested that, but it's at least closer to being correct.
•
written
by
Devon Ryan
- "Still not working" isn't something that anyone can help you with.
- Try a python forum.
•
written
by
Devon Ryan
Log in to answer this question.
Try omitting the
http://part in the URL you supply tohttplib.HTTPConnection(). The method seems to split by:and use the part after it as the port number.I have made changes, that is
but I am getting the error:
Please assist.