How to make a POST with the Python Requests module?

The Requests Python package is the first choice when we want to perform HTTP post requests in Python. I use it every day in my web-scraping / data-collection journey.

This guide will cover everything you need to know to master sending data with the POST HTTP request.

Quick example: performing a POST request with the Requests module

Here, we make a simple POST request by sending JSON data, and then we print the response.

Python

    import requests

url = "https://httpbin.org/post"
my_data = {"john": "doe"}

response = requests.post(url, json=my_data)

print(response.json())

The output should look like this:

Bash

    {'args': {}, 'data': '{"john": "doe"}', 'files': {}, 'form': {}, 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Content-Length': '15', 'Content-Type': 'application/json', 'Host': 'httpbin.org', 'User-Agent': 'python-requests/2.32.3', 'X-Amzn-Trace-Id': 'Root=1-66eafda9-03c9f45c5ba30bca01a205c5'}, 'json': {'john': 'doe'}, 'origin': 'IP-CENSORED', 'url': 'https://httpbin.org/post'}

POST request with `JSON` argument

We will cover the json parameter's specifics. It is often used to write data when communicating with an API endpoint.

When communicating with APIs that expect JSON data, you should set some headers like:

We will cover the json parameter's specifics. It is often used to write data when communicating with an API endpoint.

When communicating with APIs that expect JSON data, you should set some headers like:

Python

    headers = {
    "Accept": "application/json",
    "Content-Type": "application/json",
}

But with this json parameter, you don’t need to do that. It sets the proper headers and encodes your dictionary or JSON object.

This gives us something straightforward like this:

Python

    headers = {
    "Accept": "application/json",
    "Content-Type": "application/json",
}

POST request form data with `DATA` argument

The data argument is useful when submitting form data or sending payloads that don’t require JSON encoding. This is common when interacting with websites that use traditional HTML forms.

The data argument is useful when submitting form data or sending payloads that don’t require JSON encoding. This is common when interacting with websites that use traditional HTML forms.

Python

    import requests

url = "https://httpbin.org/post"
data = {"username": "jeremylp", "password": "strongpassword"}

response = requests.post(url, data=data)

print(response.text)

This sends the data in application/x-www-form-urlencoded format, the default format for form submissions.

Here is the output:

Bash

    {
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {
    "password": "strongpassword", 
    "username": "jeremylp"
  }, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate", 
    "Content-Length": "41", 
    "Content-Type": "application/x-www-form-urlencoded", 
    "Host": "httpbin.org", 
    "User-Agent": "python-requests/2.32.3", 
    "X-Amzn-Trace-Id": "Root=1-66eb33aa-7af65511275545d24f8a9f17"
  }, 
  "json": null, 
  "origin": "IP_CENSORED", 
  "url": "https://httpbin.org/post"
}

Defining Content-Type & Content-Encoding

In this short section, we’ll see an example of how to specify the data format and the data encoding.

Suppose we have a heavy JSON to send and use the gzip compressing to optimize the internet traffic.

💡 GZIP is a compression technology frequently used to send and receive data quickly over the internet.

In the following example, we will compress the dictionary {”heavy”: “data”}, using gzip, to send it faster to the endpoint.

Python

    import requests
import gzip
import json

url = "https://enxsgnb905ynf.x.pipedream.net"
data = {"heavy": "data"}

# The gzip.compress(...) method takes `bytes` in argument
json_data = json.dumps(data).encode("utf-8")
gzipped_data = gzip.compress(json_data)


# Here we define the headers including 'Content-Encoding'
# to signal gzip compression
headers = {
    "Content-Type": "application/json",  # The data format
    "Content-Encoding": "gzip",  # The data is gzipped
}

# Perform the POST request
response = requests.post(url, data=gzipped_data, headers=headers)

print(response.text)

In this code example, we use the compress method from the gzip module for compression.

Which takes in argument, a bytes object. That’s why we have to first encode the dict like this: json_data = json.dumps(data).encode("utf-8").

Now that our json_data is bytes, we can use gzip: gzipped_data = gzip.compress(json_data).

At this point, our data is ready to be sent. But, before performing the request, we must set some headers. They tell the server it is receiving JSON data, encoded and compressed with gzip.

So we set the header "Content-Type": "application/json" to specify that the content format is JSON.

Then, the header "Content-Encoding": "gzip" tells the server to decode the binary data using gzip.

Here is what we received in the request bin page:

example of results of a POST request using Python request

We can see that the server successfully read our {”heavy”: “data”}.

Setting cookies with POST requests method

You can also perform POST HTTP requests by passing a cookie dictionary. For example, this is needed to submit a form as an authenticated user.

Python

    import requests

url = "https://httpbin.org/post"
cookies = {"session_id": "705897e9c4cb41128a272b653bd2e65d"}

response = requests.post(url, json={"full_name": "John Doe"}, cookies=cookies)

Performing POST request using the Requests Session object

This last section will show the simple syntax for using the Session object to perform POST requests.

It’s efficient to reuse the same session object when making multiple requests to the same server.

The session object maintains parameters such as cookies and headers across multiple requests.

Python

    from requests import Session

url = "https://httpbin.org/post"
headers = {"Content-Type": "application/json"}
data = {"session": "object"}

session = Session()
session.headers.update(headers)

response = session.post(url, data=data)

In this short example, we create a Session object. We import it from the requests library: from requests import Session.

Then, we update the session headers, with Content-Type set to application/json: session.headers.update(headers).

By doing that, we don't need to specify the headers in the requests we perform with the session.

Finally, we perform the request: session.post(url, data=data), which is similar to the requests.post(...) syntax we saw before.

Definition and usage of the POST HTTP method

The POST HTTP method is used when you want to send data inside the request's body.

The post(…) function sends an HTTP request with the POST method. It allows you to encode the request's body content as needed automatically.

Overview of other HTTP requests

If you're not familiar with the HTTP protocol, note that it has different methods for different purposes.

The two most common HTTP methods are the GET and POST methods. Yet, other HTTP methods are helpful in the context of REST APIs.

GET: The most used HTTP method. It’s used to retrieve data. You can also send data directly to the URL using this HTTP method. It’s called query params.
POST: As we discussed, the POST method is used to send data or perform a “create” action.
PUT: The PUT HTTP method is similar to the POST; it performs an “update” action.
PATCH: Here, it also performs an update but a “partial update” action.
DELETE: The name is quite explicit. It is to perform a “delete” action.
OPTIONS: A less common method. Some API endpoints respond to it. They provide info on how to use the endpoint.
HEAD: The usage is quite specific here. It retrieves the metadata of a resource from a server, such as a file name or the file size stored in a server.

Here is a small example of how to use each of those HTTP methods with the Requests package:

Python

    import requests

resp = requests.get("https://httpbin.org/get", params={"query": "params"})
resp = requests.post("https://httpbin.org/post", json={"john": "doe"})
resp = requests.put("https://httpbin.org/put", data={"john": "doe"})
resp = requests.delete("https://httpbin.org/delete")
resp = requests.options("https://httpbin.org/options")
resp = requests.head("https://httpbin.org/head")

requests.post() parameters

Here are all the parameters you can use with the post(…) method:

Parameter	Is required	Description
url	Required	The URL to request
data	Optional	A dictionary, list of tuples, bytes, or file-like object to send in the body to the specified url
json	Optional	A JSON serializable Python object to send in the body of the request
headers	Optional	A dictionary of HTTP headers to send with the request
cookies	Optional	A dictionary or CookieJar object to send with the request
files	Optional	A dictionary of {'name': file-like-objects} (or {'name': file-tuple}) for multipart encoding upload.
auth	Optional	A tuple to enable a certain HTTP authentication.Default None
timeout	Optional	A float, or a tuple, how many seconds to wait for the server to send data before giving up, as a float. As a tuple: (<connect timeout>, <read timeout>) Default None which means the request will continue until the connection is closed
allow_redirects	Optional, default: True	A boolean to enable/disable redirection.
proxies	Optional	A dictionary of the protocol to the proxy url.
stream	Optional, default: False	A boolean indication if the response should be immediately downloaded (False) or streamed (True).Default False
verify	Optional, default: True	Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use. Default True
cert	Optional	If string, path to ssl client cert file (.pem). If tuple, ('cert', 'key') pair. Default None

The return value of all HTTP requests made with the Requests package is a Response object.

How to make a POST with the Python Requests package?

Quick example: performing a POST request with the Requests module

POST request with JSON argument

POST request form data with DATA argument

Defining Content-Type & Content-Encoding

Setting cookies with POST requests method

Performing POST request using the Requests Session object

Definition and usage of the POST HTTP method

Overview of other HTTP requests

requests.post() parameters

POST request with `JSON` argument

POST request form data with `DATA` argument