User Guide¶
Making your first HTTP request¶
First things first, import the hip module:
>>> import hip
You’ll need a PoolManager instance to make requests.
This object handles all of the details of connection pooling and thread safety
so that you don’t have to:
>>> http = hip.PoolManager()
To make a request use request():
>>> r = http.request('GET', 'http://httpbin.org/robots.txt')
>>> r.data
b'User-agent: *\nDisallow: /deny\n'
request() returns a HTTPResponse object, the
Response content section explains how to handle various responses.
You can use request() to make requests using any
HTTP verb:
>>> r = http.request(
... 'POST',
... 'http://httpbin.org/post',
... fields={'hello': 'world'})
The Request Data section covers sending other kinds of requests data, including JSON, files, and binary data.
Response content¶
The HTTPResponse object provides
status, data, and
header attributes:
>>> r = http.request('GET', 'http://httpbin.org/ip')
>>> r.status
200
>>> r.data
b'{\n "origin": "104.232.115.37"\n}\n'
>>> r.headers
HTTPHeaderDict({'Content-Length': '33', ...})
JSON content¶
JSON content can be loaded by decoding and deserializing the
data attribute of the request:
>>> import json
>>> r = http.request('GET', 'http://httpbin.org/ip')
>>> json.loads(r.data.decode('utf-8'))
{'origin': '127.0.0.1'}
Binary content¶
The data attribute of the response is always set
to a byte string representing the response content:
>>> r = http.request('GET', 'http://httpbin.org/bytes/8')
>>> r.data
b'\xaa\xa5H?\x95\xe9\x9b\x11'
Note
For larger responses, it’s sometimes better to stream the response.
Using io Wrappers with Response content¶
Sometimes you want to use io.TextIOWrapper or similar objects like a CSV reader
directly with HTTPResponse data. Making these two interfaces play nice
together requires using the auto_close attribute by setting it
to False. By default HTTP responses are closed after reading all bytes, this disables that behavior:
>>> import io
>>> r = http.request('GET', 'https://example.com', preload_content=False)
>>> r.auto_close = False
>>> for line in io.TextIOWrapper(r):
>>> print(line)
Request Data¶
Headers¶
You can specify headers as a dictionary in the headers argument in request():
>>> r = http.request(
... 'GET',
... 'http://httpbin.org/headers',
... headers={
... 'X-Something': 'value'
... })
>>> json.loads(r.data.decode('utf-8'))['headers']
{'X-Something': 'value', ...}
Query Parameters¶
For GET, HEAD, and DELETE requests, you can simply pass the
arguments as a dictionary in the fields argument to
request():
>>> r = http.request(
... 'GET',
... 'http://httpbin.org/get',
... fields={'arg': 'value'})
>>> json.loads(r.data.decode('utf-8'))['args']
{'arg': 'value'}
For POST and PUT requests, you need to manually encode query parameters
in the URL:
>>> from urllib.parse import urlencode
>>> encoded_args = urlencode({'arg': 'value'})
>>> url = 'http://httpbin.org/post?' + encoded_args
>>> r = http.request('POST', url)
>>> json.loads(r.data.decode('utf-8'))['args']
{'arg': 'value'}
Form Data¶
For PUT and POST requests, hip will automatically form-encode the
dictionary in the fields argument provided to
request():
>>> r = http.request(
... 'POST',
... 'http://httpbin.org/post',
... fields={'field': 'value'})
>>> json.loads(r.data.decode('utf-8'))['form']
{'field': 'value'}
JSON¶
You can send a JSON request by specifying the encoded data as the body
argument and setting the Content-Type header when calling
request():
>>> import json
>>> data = {'attribute': 'value'}
>>> encoded_data = json.dumps(data).encode('utf-8')
>>> r = http.request(
... 'POST',
... 'http://httpbin.org/post',
... body=encoded_data,
... headers={'Content-Type': 'application/json'})
>>> json.loads(r.data.decode('utf-8'))['json']
{'attribute': 'value'}
Files & Binary Data¶
For uploading files using multipart/form-data encoding you can use the same
approach as Form Data and specify the file field as a tuple of
(file_name, file_data):
>>> with open('example.txt') as fp:
... file_data = fp.read()
>>> r = http.request(
... 'POST',
... 'http://httpbin.org/post',
... fields={
... 'filefield': ('example.txt', file_data),
... })
>>> json.loads(r.data.decode('utf-8'))['files']
{'filefield': '...'}
While specifying the filename is not strictly required, it’s recommended in order to match browser behavior. You can also pass a third item in the tuple to specify the file’s MIME type explicitly:
>>> r = http.request(
... 'POST',
... 'http://httpbin.org/post',
... fields={
... 'filefield': ('example.txt', file_data, 'text/plain'),
... })
For sending raw binary data simply specify the body argument. It’s also
recommended to set the Content-Type header:
>>> with open('example.jpg', 'rb') as fp:
... binary_data = fp.read()
>>> r = http.request(
... 'POST',
... 'http://httpbin.org/post',
... body=binary_data,
... headers={'Content-Type': 'image/jpeg'})
>>> json.loads(r.data.decode('utf-8'))['data']
b'...'
Certificate Verification¶
While you can disable certification verification, it is highly recommend to leave it on.
Unless otherwise specified hip will try to load the default system certificate stores. The most reliable cross-platform method is to use the certifi package which provides Mozilla’s root certificate bundle:
python -m pip install certifi
Once you have certificates, you can create a PoolManager
that verifies certificates when making requests:
>>> import certifi
>>> import hip
>>> http = hip.PoolManager(
... cert_reqs='CERT_REQUIRED',
... ca_certs=certifi.where())
The PoolManager will automatically handle certificate
verification and will raise SSLError if verification fails:
>>> http.request('GET', 'https://google.com')
(No exception)
>>> http.request('GET', 'https://expired.badssl.com')
hip.exceptions.SSLError ...
Note
You can use OS-provided certificates if desired. Just specify the full
path to the certificate bundle as the ca_certs argument instead of
certifi.where(). For example, most Linux systems store the certificates
at /etc/ssl/certs/ca-certificates.crt. Other operating systems can
be difficult.
Using Timeouts¶
Timeouts allow you to control how long (in seconds) requests are allowed to run
before being aborted. In simple cases, you can specify a timeout as a float
to request():
>>> http.request(
... 'GET', 'http://httpbin.org/delay/3', timeout=4.0)
<hip.response.HTTPResponse>
>>> http.request(
... 'GET', 'http://httpbin.org/delay/3', timeout=2.5)
MaxRetryError caused by ReadTimeoutError
For more granular control you can use a Timeout
instance which lets you specify separate connect and read timeouts:
>>> http.request(
... 'GET',
... 'http://httpbin.org/delay/3',
... timeout=hip.Timeout(connect=1.0))
<hip.response.HTTPResponse>
>>> http.request(
... 'GET',
... 'http://httpbin.org/delay/3',
... timeout=hip.Timeout(connect=1.0, read=2.0))
MaxRetryError caused by ReadTimeoutError
If you want all requests to be subject to the same timeout, you can specify
the timeout at the PoolManager level:
>>> http = hip.PoolManager(timeout=3.0)
>>> http = hip.PoolManager(
... timeout=hip.Timeout(connect=1.0, read=2.0))
You still override this pool-level timeout by specifying timeout to
request().
Retrying Requests¶
hip can automatically retry idempotent requests. This same mechanism also
handles redirects. You can control the retries using the retries parameter
to request(). By default, hip will retry
requests 3 times and follow up to 3 redirects.
To change the number of retries just specify an integer:
>>> http.request('GET', 'http://httpbin.org/ip', retries=10)
To disable all retry and redirect logic specify retries=False:
>>> http.request(
... 'GET', 'http://nxdomain.example.com', retries=False)
NewConnectionError
>>> r = http.request(
... 'GET', 'http://httpbin.org/redirect/1', retries=False)
>>> r.status
302
To disable redirects but keep the retrying logic, specify redirect=False:
>>> r = http.request(
... 'GET', 'http://httpbin.org/redirect/1', redirect=False)
>>> r.status
302
For more granular control you can use a Retry instance.
This class allows you far greater control of how requests are retried.
For example, to do a total of 3 retries, but limit to only 2 redirects:
>>> http.request(
... 'GET',
... 'http://httpbin.org/redirect/3',
... retries=hip.Retry(3, redirect=2))
MaxRetryError
You can also disable exceptions for too many redirects and just return the
302 response:
>>> r = http.request(
... 'GET',
... 'http://httpbin.org/redirect/3',
... retries=hip.Retry(
... redirect=2, raise_on_redirect=False))
>>> r.status
302
If you want all requests to be subject to the same retry policy, you can
specify the retry at the PoolManager level:
>>> http = hip.PoolManager(retries=False)
>>> http = hip.PoolManager(
... retries=hip.Retry(5, redirect=2))
You still override this pool-level retry policy by specifying retries to
request().
Errors & Exceptions¶
hip wraps lower-level exceptions, for example:
>>> try:
... http.request('GET', 'nx.example.com', retries=False)
>>> except hip.exceptions.NewConnectionError:
... print('Connection failed.')
See exceptions for the full list of all exceptions.