Sep. 21, 2012

Urllib2 - User Agent


This post will show how to add headers to a HTTP request.

By default urllib2 identifies itself as Python-urllib/2.7 : GET / HTTP/1.1" 200
151 "-" "Python-urllib/2.7"

That can sometimes be confusing for certain sites. With the user_agent header in
Python, it's possible to alter that and specify any identity you like. 
The example below, use the Mozilla 5.10 as a User Agent, and that is also what
will show up in the web server log file. 
import urllib2

req = urllib2.Request('')

req.add_header('User-agent', 'Mozilla 5.10')

res = urllib2.urlopen(req)

html =
This is what will show up in the log file.
"GET / HTTP/1.1" 200 151 "-" "Mozilla 5.10"

Recommended Python Training – DataCamp

For Python training, our top recommendation is DataCamp.

Datacamp provides online interactive courses that combine interactive coding challenges with videos from top instructors in the field.

Datacamp has beginner to advanced Python training that programmers of all levels benefit from.


Download Our Free Guide To Learning Python

* indicates required

Read more about:
Disclosure of Material Connection: Some of the links in the post above are “affiliate links.” This means if you click on the link and purchase the item, I will receive an affiliate commission. Regardless, only recommend products or services that we try personally and believe will add value to our readers.