What is Requests?
Requests is an Apache2 Licensed HTTP library, written in Python, for human beings
Requests allow you to send HTTP/1.1 requests.
You can add headers, form data, multipart files, and parameters with simple
Python dictionaries, and access the response data in the same way.
Install Requests
There are a few ways to install Requests. Either use pip, easy_install,
or get the tarball.
You can also get the code through GitHub, if you prefer that.
To see the full list of options, please consult the install documentation here.
We are using pip to install, simply type in:
pip install requests
Importing the module
To import the Requests module, put this command at the beginning of your script.
import requests
Making a request
Note, I will sometimes use http://httpbin.org/ throughout this article.
It supports most of the common HTTP verbs and mostly return the variables you
send in, which is sometimes very useful when you are writing a (HTTP) script.
# Get a webpage, this creates a Response object called "r"
r = requests.get('https://github.com/timeline.json')
Response Code
We can check the response status code, and do a status code lookup with the
dictionary look-up object.
r = requests.get('https://github.com/timeline.json')
r.status_code
>>200
r.status_code == requests.codes.ok
>>> True
requests.codes['temporary_redirect']
>>> 307
requests.codes.teapot
>>> 418
requests.codes['\o/']
>>> 200
Get the content
Get the content of the server's response.
import requests
r = requests.get('https://github.com/timeline.json')
print r.text
# Requests also comes with a builtin JSON decoder,
# in case you’re dealing with JSON data
import requests
r = requests.get('https://github.com/timeline.json')
print r.json
Headers
We can view the server’s response headers using a Python dictionary, and we can
access the headers using any capitalization we want.
If a header doesn't exist in the Response, its value defaults to None
r.headers
{
'status': '200 OK',
'content-encoding': 'gzip',
'transfer-encoding': 'chunked',
'connection': 'close',
'server': 'nginx/1.0.4',
'x-runtime': '148ms',
'etag': '"e1ca502697e5c9317743dc078f67693f"',
'content-type': 'application/json; charset=utf-8'
}
r.headers['Content-Type']
>>>'application/json; charset=utf-8'
r.headers.get('content-type')
>>>'application/json; charset=utf-8'
r.headers['X-Random']
>>>None
# Get the headers of a given URL
resp = requests.head("http://www.google.com")
print resp.status_code, resp.text, resp.headers
Encoding
Requests will automatically decode content from the server.
Most Unicode charsets are seamlessly decoded.
When you make a request, Requests makes educated guesses about the encoding
of the response based on the HTTP headers.
The text encoding guessed by Requests is used when you access r.text.
You can find out what encoding Requests is using, and change it, using the
r.encoding property:
If you change the encoding, Requests will use the new value of r.encoding
whenever you call r.text.
print r.encoding
>> utf-8
>>> r.encoding = 'ISO-8859-1'
Custom Headers
If you’d like to add HTTP headers to a request, simply pass in a dict to the
headers parameter.
import json
url = 'https://api.github.com/some/endpoint'
payload = {'some': 'data'}
headers = {'content-type': 'application/json'}
r = requests.post(url, data=json.dumps(payload), headers=headers)
Redirection and History
Requests will automatically perform location redirection while using the GET
and OPTIONS verbs.
GitHub redirects all HTTP requests to HTTPS.
We can use the history method of the Response object to track redirection.
r = requests.get('http://github.com')
r.url
>>> 'https://github.com/'
r.status_code
>>> 200
r.history
>>> []
Make a HTTP Post request
With Requests you can of course also do post requests.
r = requests.post("http://httpbin.org/post")
You can use other HTTP requests types as well (PUT, DELETE, HEAD and OPTIONS)
r = requests.put("http://httpbin.org/put")
r = requests.delete("http://httpbin.org/delete")
r = requests.head("http://httpbin.org/get")
r = requests.options("http://httpbin.org/get")
# This small script creates a Github repo.
import requests, json
github_url = "https://api.github.com/user/repos"
data = json.dumps({'name':'test', 'description':'some test repo'})
r = requests.post(github_url, data, auth=('user', '*****'))
print r.json
Errors and Exceptions
In the event of a network problem (e.g. DNS failure, refused connection, etc),
Requests will raise a ConnectionError exception.
In the event of the rare invalid HTTP response, Requests will raise an HTTPError
exception.
If a request times out, a Timeout exception is raised.
If a request exceeds the configured number of maximum redirections, a
TooManyRedirects exception is raised.
All exceptions that Requests explicitly raises inherit from
requests.exceptions.RequestException.
.png)
