Python HTTP Request Tutorial

By: Rajesh P.S.

HTTP stands for Hypertext Transfer Protocol. It is an application-layer protocol used for transmitting data over the internet. HTTP is the foundation of data communication for the World Wide Web, and it enables the communication between web servers and web clients (usually web browsers).

Request-Response protocol

HTTP works as a request-response protocol between a client and a server. When a client (such as a web browser) wants to access a resource (such as a web page) on a server, it sends an HTTP request to the server. The server then responds with an HTTP response, which contains the requested resource (such as an HTML document) or an error message if the request cannot be fulfilled.

Stateless Protocol

HTTP is a stateless protocol, which means that each request and response pair is independent and does not carry any information about previous requests or responses. However, HTTP cookies and other mechanisms can be used to maintain state and session information across multiple requests and responses.

HTTP methods

HTTP methods, also known as HTTP verbs, are the actions that can be performed on resources identified by a Uniform Resource Identifier (URI) in an HTTP request. The most common HTTP methods are:

GET: Retrieves a representation of the specified resource. GET requests should only retrieve data and should not modify the server state.

POST: Submits an entity to the specified resource, often causing a change in state or side effects on the server.
PUT: Replaces the specified resource with the entity provided in the request payload.
DELETE: Deletes the specified resource.
HEAD: Retrieves the headers of the specified resource, but not the resource itself.
OPTIONS: Returns the HTTP methods that the server supports for the specified resource.
PATCH: Applies partial modifications to a resource.

HTTP methods are an important part of the HTTP protocol and are used to perform different types of actions on web resources, such as retrieving data, submitting data, modifying data, and deleting data.

Http status code

HTTP status codes are three-digit numbers returned by a web server in response to a client's request to indicate the outcome of the request. Following are some commonly encountered HTTP status codes and their meanings:

200 OK: The request has succeeded.
201 Created: The request has been fulfilled and a new resource has been created.
400 Bad Request: The server cannot or will not process the request due to an apparent client error.
401 Unauthorized: The request requires authentication.
403 Forbidden: The server understood the request, but is refusing to fulfill it.
404 Not Found: The requested resource could not be found but may be available in the future.
500 Internal Server Error: A generic error message, given when an unexpected condition was encountered and no more specific message is suitable.

HTTP status codes are useful for diagnosing errors and troubleshooting web applications.

Python request module

The requests module in Python is a third-party library used for making HTTP requests. It simplifies the process of sending HTTP/1.1 requests and handling the response received. The requests module is built on top of the urllib3 module and provides an easy-to-use interface for working with HTTP requests and responses.

The requests module provides several HTTP methods, such as GET, POST, PUT, DELETE, and others. It also provides various features like authentication, timeouts, proxies, SSL verification, redirection handling, and more.

Making a GET request from Python

To make an HTTP GET request in Python, you can use the requests library. Following is an example of how to use requests to make a simple HTTP GET request:

import requests response = requests.get('http://example.com') print(response.status_code) print(response.content)

In the above code, first import the requests library. Then, make a GET request to the URL http://example.com using the requests.get() method. The response from the server is stored in the response variable.

You can access the status code of the response using the status_code attribute of the response object. In this case, print the status code to the console using the print() function.

You can also access the content of the response using the content attribute of the response object. In this case, print the content to the console using the print() function.

If the server returns a response with a content type of "text/html", the content attribute will contain the HTML content of the web page. If the content type is "application/json", the content attribute will contain a JSON string that can be parsed into a Python dictionary.

You can also pass parameters to the requests.get() method to customize the HTTP request. For example, you can pass a dictionary of query parameters to the params parameter to include them in the URL:

import requests params = {'q': 'Python programming'} response = requests.get('http://example.com/search', params=params) print(response.status_code) print(response.content)

In the above example, pass a dictionary of query parameters to the params parameter. The resulting URL will be http://example.com/search?q=Python+programming. The server will then return the search results for the query "Python programming".

The requests library also provides many other features for making HTTP requests, such as setting headers, handling redirects, and handling authentication.

Making a POST request from Python

To make an HTTP POST request in Python, you can use the requests library. Following is an example of how to use requests to make a simple HTTP POST request:

import requests payload = {'key1': 'value1', 'key2': 'value2'} response = requests.post('http://example.com/post', data=payload) print(response.status_code) print(response.content)

In the above code, first import the requests library. Then, create a dictionary called payload that contains the data you want to send in the POST request. Then use the requests.post() method to make the POST request to the URL http://example.com/post, passing the data parameter with the payload dictionary.

The response from the server is stored in the response variable. You can access the status code of the response using the status_code attribute of the response object. In this case, print the status code to the console using the print() function.

You can also access the content of the response using the content attribute of the response object. In this case, print the content to the console using the print() function.

You can also pass JSON data to the requests.post() method using the json parameter instead of the data parameter.

import requests payload = {'key1': 'value1', 'key2': 'value2'} response = requests.post('http://example.com/post', json=payload) print(response.status_code) print(response.content)

In the above code, create the same payload dictionary as in the previous example. Then use the requests.post() method to make the POST request, passing the json parameter with the payload dictionary. This tells requests to send the data as a JSON-encoded request body.

Also, you can access the status code and content of the response using the status_code and content attributes of the response object, respectively.

The requests library also provides many other features for making HTTP requests, such as setting headers, handling redirects, and handling authentication. You can find more information about these features in the requests documentation.

Exception Handling Of Python Requests Module

Handling errors is an important aspect of making HTTP requests in Python. The requests module provides several ways to handle errors that may occur during HTTP requests.

Following is an example of how to handle errors in requests:

import requests try: response = requests.get('https://www.example.com') response.raise_for_status() # This will raise an HTTPError if the response status code is not in the 200-299 range. except requests.exceptions.HTTPError as error: print(f"HTTP Error occurred: {error}") except requests.exceptions.Timeout as error: print(f"Timeout Error occurred: {error}") except requests.exceptions.ConnectionError as error: print(f"Connection Error occurred: {error}") except requests.exceptions.RequestException as error: print(f"Unknown Error occurred: {error}") else: print(response.text)

In the above example, first make a GET request to https://www.example.com using requests.get(). Then use a try-except block to handle errors that may occur during the request.

If the request returns a status code in the 200-299 range, the else block is executed, and print the content of the response using the response.text attribute.

If an error occurs during the request, one of the exception blocks will be executed depending on the type of error. The HTTPError exception is raised if the response status code is not in the 200-299 range. You can also use the raise_for_status() method on the response object to explicitly raise an HTTPError if the status code is not in the 200-299 range.

The Timeout exception is raised if the request times out. The ConnectionError exception is raised if a connection cannot be established. The RequestException exception is a catch-all for any other exception that may occur during the request.

By handling errors properly, you can make our HTTP requests more robust and prevent our code from crashing or behaving unexpectedly when errors occur.

Python http request without library

It is possible to make an HTTP request in Python without using Requests or any third-party library or other external dependencies. Followiong is a basic example of making an HTTP GET request using only Python's built-in modules:

import socket HOST = 'www.example.com' PORT = 80 request = f"GET / HTTP/1.1\r\nHost: {HOST}\r\n\r\n" with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: s.connect((HOST, PORT)) s.sendall(request.encode()) response = b'' while True: data = s.recv(1024) if not data: break response += data print(response.decode())

This code establishes a TCP connection to the given host and port, sends an HTTP GET request, and receives the response. Note that this example only handles the simplest case of an HTTP request and response; handling more complex cases, such as HTTP redirects or authentication, would require additional code. Also, keep in mind that using third-party libraries, such as the popular requests library, can simplify the process and provide additional functionality.

NEXT..... Difference between Set and List in Python