Entry

Specifying a proxy server to be used for httplib?

Aug 2nd, 2000 04:25
unknown unknown, Amit, David Currie

Problem:
The Internet in our office can only be accessed through a proxy server.
I wanted to write a utility in Python that uses the httplib module to
automate some routine searches. But I am stuck since there does not
seem to be a way to access the Internet thru' the proxy.
I am sure there is a very small thing which I am missing. Could someone
please advise on how to channel the request thru' proxy and automate
authentication?
Solution:
I figured it out by reading RFC 2617 on "HTTP Authentication: Basic and
Digest Access Authentication", and by a helpful post by David Currie
dated 12 July, 2000.
The solution - in this case - was as follows:
---------------------------------------------
import httplib
import base64
# 1. connect to the proxy
h1 = httplib.HTTP("154.112.170.19:80")
# 2. give the absolute URL in the request
h1.putrequest('GET', 'http://www.yahoo.com/')
h1.putheader('Accept', 'text/html')
h1.putheader('Accept', 'text/plain')
# 3. set the header with a base64 encoding of user-id:passwd
auth = "Basic " + base64.encodestring("john_doe:glug#123")
h1.putheader('Proxy-Authorization', auth)
h1.endheaders()
# 4. get the page
errcode, errmsg, headers = h1.getreply()
print errcode
print errmsg
print headers
f=h1.getfile()
for line in f.readlines():
    print line
---------------------------------------------