Calling APIs from Python with requests

In Python for AI you made a first requests.get call. This page turns that into a skill. You have used curl to talk to the Practice API; now you will do the same from Python, where the answers become dictionaries, lists and variables you can loop over, clean, store and feed into a model. Almost every AI product does this somewhere: it fetches data, calls a model's API, or calls another team's service.

The aim is not only to make a call that works today. It is to write calls that behave well when the network is slow, the server is down, the key is missing or the answer is not what you expected. Those four cases are where beginners' scripts hang forever, crash with a wall of traceback, or silently save garbage.

What you will learn

• Install requests and make GET and POST calls with query parameters, headers and JSON bodies.
• Read a response: status code, headers, text and JSON, and decide success or failure with raise_for_status().
• Handle failures with the exceptions in requests.exceptions, and why every call needs a timeout.
• Keep API keys in environment variables, never in code.
• Read a paginated API (next_page) completely with a small loop or generator.

Setting up

Keep the Practice API running in its second terminal (fastapi run practice_api.py --port 8800, from Web APIs and REST: Reading Docs and Testing with curl). In your first terminal make a project with a virtual environment and install requests:

mkdir -p ~/projects/api-client
cd ~/projects/api-client
python -m venv .venv
source .venv/bin/activate
pip install -q requests

With uv the same is uv init then uv add requests. On Windows PowerShell, activate with .venv\Scripts\Activate.ps1. The examples below are Python; type them into a file and run it with python file.py, or into a Python REPL or a notebook. They share variables from one block to the next, like a notebook.

Your first GET

import requests

BASE = "http://localhost:8800"

response = requests.get(f"{BASE}/health", timeout=5)
print(response.status_code)
print(response.headers["content-type"])
print(response.text)
print(response.json())

200
application/json
{"status":"ok"}
{'status': 'ok'}

requests.get(url) sends one GET request and returns a Response object. It holds everything the server sent back. The attributes you will use all the time:

Notice timeout=5. We will come back to it properly; for now remember that it is part of every call you write.

Query parameters

You could build ?level=beginner&per_page=3 by hand in an f-string, but then you must URL-encode special characters yourself. Pass a dictionary to params instead and requests builds and encodes the query string:

response = requests.get(
    f"{BASE}/v1/courses",
    params={"level": "beginner", "per_page": 3},
    timeout=5,
)
print(response.url)
data = response.json()
print(data["total"], data["page"], data["next_page"])
for course in data["items"]:
    print(course["id"], course["title"])

http://localhost:8800/v1/courses?level=beginner&per_page=3
5 1 2
1 How AI Works
2 Python for AI
3 Maths for AI

response.url is a quick way to check what you really asked for. A value of None in params is dropped, and a list becomes a repeated key (?tag=a&tag=b). Try a city with a space or a non-English letter and look at response.url:

response = requests.get(f"{BASE}/v1/weather", params={"city": "Cox's Bazar"}, timeout=5)
print(response.url)
print(response.status_code, response.json())

http://localhost:8800/v1/weather?city=Cox%27s+Bazar
404 {'detail': 'No weather data for "Cox\'s Bazar"'}

Reading JSON

response.json() returns normal Python objects: JSON objects become dict, arrays become list, null becomes None. That means everything you learned about dictionaries and comprehensions applies:

response = requests.get(f"{BASE}/v1/courses/2", timeout=5)
course = response.json()
print(course)
print(course["title"], "-", course["hours"], "hours")

listing = requests.get(f"{BASE}/v1/courses", params={"per_page": 20}, timeout=5).json()["items"]
short = [c["title"] for c in listing if c["hours"] <= 6]
print(short)

{'id': 2, 'title': 'Python for AI', 'level': 'beginner', 'hours': 12, 'rating': 4.8}
Python for AI - 12 hours
['How AI Works', 'Prompt Engineering', 'LLM APIs']

Two habits protect you from surprises. First, use course.get("rating") (returns None if missing) when a field may be absent, and course["title"] when its absence would be a bug you want to see. Second, never call .json() on a response you have not checked: an error page from a proxy is HTML, and .json() on it raises an exception.

Status codes and raise_for_status

A 404 or 500 is not an exception in requests. The call succeeded at the network level; the server simply said "no". So you must look at the status yourself:

response = requests.get(f"{BASE}/v1/courses/99", timeout=5)
print(response.status_code, response.ok)
print(response.json())

404 False
{'detail': 'Course 99 not found'}

The shortest safe pattern is response.raise_for_status(). It does nothing for 2xx and 3xx, and raises requests.exceptions.HTTPError for 4xx and 5xx:

try:
    response.raise_for_status()
except requests.exceptions.HTTPError as error:
    print("HTTPError:", error)
    print("status:", error.response.status_code)

HTTPError: 404 Client Error: Not Found for url: http://localhost:8800/v1/courses/99
status: 404

The exception message holds the status and URL, and error.response is the full response, so you can still read the server's explanation (error.response.json()["detail"]). When to use which:

Sending data: POST with JSON

To create a review you POST JSON to /v1/reviews, which also needs an API key in the X-API-Key header. Pass a dictionary to json=: requests converts it to JSON text and sets Content-Type: application/json for you. Pass headers as a dictionary too:

review = {"course_id": 2, "rating": 5, "comment": "Clear and practical"}
headers = {"X-API-Key": "practice-key-123"}

response = requests.post(f"{BASE}/v1/reviews", json=review, headers=headers, timeout=5)
print(response.status_code)
print(response.json())

201
{'id': 1, 'course_id': 2, 'rating': 5, 'comment': 'Clear and practical'}

Here the key is typed in the code only to show the header; two sections down you will see the right way. First, what does json= really send? /v1/echo shows it, and also the difference from data=, which sends a form body:

as_json = requests.post(f"{BASE}/v1/echo", json={"name": "Nadia"}, timeout=5).json()
print(as_json["headers"]["content-type"], as_json["json"])

as_form = requests.post(f"{BASE}/v1/echo", data={"name": "Nadia"}, timeout=5).json()
print(as_form["headers"]["content-type"], as_form["json"])

application/json {'name': 'Nadia'}
application/x-www-form-urlencoded name=Nadia

A modern JSON API expects json=. Using data= with a dictionary is a classic mistake: the server receives a form, not JSON, and answers 422 or 400. The same call without a key, and with bad data, shows how failures look:

no_key = requests.post(f"{BASE}/v1/reviews", json=review, timeout=5)
print(no_key.status_code, no_key.json())

bad = requests.post(f"{BASE}/v1/reviews", json={**review, "rating": 9}, headers=headers, timeout=5)
print(bad.status_code)
problem = bad.json()["detail"][0]
print(problem["loc"], problem["msg"])

401 {'detail': 'Missing API key: send it in the X-API-Key header'}
422
['body', 'rating'] Input should be less than or equal to 5

401 means "who are you?" (missing key), and 422 means "I understood the JSON, but the values are invalid", with the problem field named in detail. Reading those messages is how you fix your request. The other methods work the same way: requests.put, requests.patch, requests.delete, and requests.request("GET", …) when the method is a variable.

What can go wrong: exceptions

An HTTP status is only one way a call can fail. Sometimes no response arrives at all: the server is off, the network is down, the name does not resolve, or the server takes too long. In these cases requests raises an exception. They all inherit from requests.exceptions.RequestException:

Let us trigger two of them on purpose. Port 8001 has nothing listening, and /v1/slow waits as long as you tell it:

try:
    requests.get("http://localhost:8001/health", timeout=2)
except requests.exceptions.ConnectionError as error:
    print("ConnectionError: could not connect")

try:
    requests.get(f"{BASE}/v1/slow", params={"seconds": 3}, timeout=1)
except requests.exceptions.Timeout as error:
    print("Timeout:", type(error).__name__)

ConnectionError: could not connect
Timeout: ReadTimeout

Always set a timeout

This is the most important rule on the page. requests has no default timeout: without one, a call to a server that accepts the connection and then goes silent can wait forever, and your script, your notebook or your whole web server worker hangs with it. Give every call a timeout in seconds:

• timeout=5 applies 5 seconds to connecting and 5 to waiting for data between bytes.
• timeout=(3.05, 30) sets connect and read separately. Connecting should be quick; an LLM answer may genuinely take longer, so allow the read part more.

The read timeout is the time the server stays silent, not the total time of the download. Choose values from what is normal: a few seconds for ordinary JSON APIs, 30–120 seconds for a non-streaming LLM call. To avoid typing it every time, define it once:

TIMEOUT = (3.05, 10)

started = requests.get(f"{BASE}/v1/slow", params={"seconds": 0.2}, timeout=TIMEOUT)
print(started.json(), started.elapsed.total_seconds() >= 0.2)

{'waited_seconds': 0.2} True

(We print only whether the wait was at least 0.2 s, so the output is the same on every computer.)

Putting it together: a safe fetch function

Now combine the pieces into one function that never hangs, turns every failure into a clear message and returns the data on success. Notice the order of the except clauses: HTTPError and Timeout first, the general RequestException last:

def fetch_json(url, **kwargs):
    """GET a URL and return its JSON, or raise RuntimeError with a readable message."""
    try:
        response = requests.get(url, timeout=TIMEOUT, **kwargs)
        response.raise_for_status()
        return response.json()
    except requests.exceptions.HTTPError as error:
        status = error.response.status_code
        raise RuntimeError(f"server said {status} for {url}") from error
    except requests.exceptions.Timeout as error:
        raise RuntimeError(f"timed out calling {url}") from error
    except requests.exceptions.ConnectionError as error:
        raise RuntimeError(f"could not connect to {url}") from error
    except requests.exceptions.JSONDecodeError as error:
        raise RuntimeError(f"{url} did not return JSON") from error
    except requests.exceptions.RequestException as error:
        raise RuntimeError(f"request failed: {type(error).__name__}") from error

print(fetch_json(f"{BASE}/v1/weather", params={"city": "Sylhet"}))
for bad_url in (f"{BASE}/v1/courses/99", "http://localhost:8001/health"):
    try:
        fetch_json(bad_url)
    except RuntimeError as error:
        print("failed:", error)

{'city': 'Sylhet', 'temp_c': 27.4, 'humidity': 85, 'condition': 'Rain'}
failed: server said 404 for http://localhost:8800/v1/courses/99
failed: could not connect to http://localhost:8001/health

Two details. raise … from error keeps the original exception attached, so a traceback shows both. And JSONDecodeError must come before the general RequestException, because it is a subclass of it (in current requests, a body that is not JSON raises requests.exceptions.JSONDecodeError); a general clause placed first would swallow it. Do not wrap everything in a bare except Exception: pass: it hides the very information you need.

API keys from environment variables

The key practice-key-123 above sat in the code. For a fake API that is harmless; for a real key (OpenAI, Anthropic, a paid weather API) it is a leak waiting to happen, because code goes to Git and GitHub (see .gitignore: Keeping Secrets and Junk Out). The rule is: secrets live in the environment, code reads them. In the terminal, before running Python:

export PRACTICE_API_KEY="practice-key-123"

(PowerShell: $env:PRACTICE_API_KEY = "practice-key-123". A .env file loaded with python-dotenv, from Environment Variables, .env Files and Virtual Environments, does the same job and is the usual way during development.) Then in Python:

import os

def get_api_key():
    key = os.environ.get("PRACTICE_API_KEY")
    if not key:
        raise RuntimeError("PRACTICE_API_KEY is not set. Export it or add it to .env")
    return key

api_key = get_api_key()
print("key loaded:", len(api_key), "characters")
print("shown safely:", api_key[:4] + "..." + api_key[-3:])

key loaded: 16 characters
shown safely: prac...123

Reading it in one place, with a clear error when it is missing, beats a mysterious 401 later. When you must log something, show only the first and last few characters, as above, and never print the whole key, never put it in a URL query string (URLs are logged by servers and proxies) and never commit it. Now the review call is clean:

headers = {"X-API-Key": get_api_key()}
response = requests.post(f"{BASE}/v1/reviews", json=review, headers=headers, timeout=TIMEOUT)
print(response.status_code, response.json()["id"])

201 2

Bearer tokens work the same way, with a different header: {"Authorization": f"Bearer {token}"}. You will use that for the chat API on the next page.

Reading paginated responses

Real APIs rarely return everything at once. /v1/courses returns per_page items (5 by default) and a next_page number, which becomes None on the last page. If you read only the first response you silently miss data, a quiet and common bug when you collect training or evaluation data. The fix is a loop that follows next_page:

def all_courses(**filters):
    """Yield every course, following next_page until it is None."""
    page = 1
    while page is not None:
        data = fetch_json(f"{BASE}/v1/courses", params={**filters, "page": page, "per_page": 3})
        yield from data["items"]
        page = data["next_page"]

titles = [course["title"] for course in all_courses()]
print(len(titles))
print(titles)
print([c["title"] for c in all_courses(level="advanced")])

10
['How AI Works', 'Python for AI', 'Maths for AI', 'SQL and Data Handling', 'Git, APIs and Backend', 'Prompt Engineering', 'LLM APIs', 'RAG Systems', 'AI Agents', 'LLMOps']
['RAG Systems', 'AI Agents', 'LLMOps']

A generator (yield) hands items out one by one, so the caller can stop early or process very large collections without holding them all in memory. Wrap it in list(...) when you want everything. Other APIs use different shapes: a next URL to follow as-is, or an opaque cursor or next_page_token to send back. The loop is the same; only the field names change, and the API's documentation tells you which. Always add a safety limit (for example a maximum page count) when you do not control the API, so a bug on its side cannot trap you in an endless loop.

Real-world use

• Data collection: pulling documents, tickets or product listings page by page into a dataset or a search index for RAG.
• Calling model providers: an LLM call is a POST with a JSON body, a bearer key from the environment, and a generous timeout. That is exactly the code on this page.
• Glue between services: your FastAPI backend (later pages) calls other services with the same functions, so a timeout or a missing key there must produce a clean error and not hang a request.

Common mistakes

• No timeout. The call can hang forever and freeze a script or a server worker. Always pass timeout=.
• Assuming 200. Calling .json() and indexing into an error body gives a confusing KeyError. Check the status or call raise_for_status() first.
• data= instead of json=. The server gets a form, not JSON, and answers with 422 or 400. Use json= for JSON APIs.
• Secrets in code or in the URL. Keys committed to Git are leaked keys. Read them from the environment and send them in headers.
• Stopping at page one. A list endpoint can have many pages. Follow next_page (or the API's equivalent) until it ends.
• Catching everything. except Exception: pass hides real bugs. Catch the specific requests.exceptions you can handle and let the rest surface.

Try it yourself

1. Easy: Get the weather for Dhaka and print one line: Dhaka: 31.5 C, Humid.
2. Easy: Fetch the three highest-rated courses: request all courses (per_page=20) and print the titles sorted by rating, best first.
3. Medium: Write course_or_none(course_id) that returns the course dict, or None for a 404, but still raises for any other error. Test with ids 3 and 99.
4. Medium: Write post_review(course_id, rating, comment) that uses the key from the environment and returns the new review's id; if the API answers 422, it should return None and print the first problem's message.
5. Hard: Using your all_courses generator, compute the total hours per level and print the levels in alphabetical order.

Answers

# 1. weather line
w = fetch_json(f"{BASE}/v1/weather", params={"city": "Dhaka"})
print(f"{w['city']}: {w['temp_c']} C, {w['condition']}")

# 2. top three by rating
courses = fetch_json(f"{BASE}/v1/courses", params={"per_page": 20})["items"]
for c in sorted(courses, key=lambda c: c["rating"], reverse=True)[:3]:
    print(c["rating"], c["title"])

Dhaka: 31.5 C, Humid
4.9 RAG Systems
4.8 Python for AI
4.8 AI Agents

# 3. None only for 404
def course_or_none(course_id):
    response = requests.get(f"{BASE}/v1/courses/{course_id}", timeout=TIMEOUT)
    if response.status_code == 404:
        return None
    response.raise_for_status()
    return response.json()

print(course_or_none(3)["title"])
print(course_or_none(99))

# 4. post_review
def post_review(course_id, rating, comment):
    response = requests.post(
        f"{BASE}/v1/reviews",
        json={"course_id": course_id, "rating": rating, "comment": comment},
        headers={"X-API-Key": get_api_key()},
        timeout=TIMEOUT,
    )
    if response.status_code == 422:
        print("invalid:", response.json()["detail"][0]["msg"])
        return None
    response.raise_for_status()
    return response.json()["id"]

print(post_review(1, 4, "Good start"))
print(post_review(1, 0, "Bad rating"))

Maths for AI
None
3
invalid: Input should be greater than or equal to 1
None

# 5. hours per level
totals = {}
for course in all_courses():
    totals[course["level"]] = totals.get(course["level"], 0) + course["hours"]
for level in sorted(totals):
    print(level, totals[level])

advanced 30
beginner 42
intermediate 11

Summary

• requests.get/post(url, params=…, json=…, headers=…, timeout=…) returns a Response with status_code, headers, text and .json().
• A 4xx or 5xx is not an exception by itself; call raise_for_status() or check status_code.
• Network failures raise requests.exceptions classes: ConnectionError, Timeout, HTTPError, all under RequestException.
• Always set a timeout; requests has none by default.
• Keep keys in environment variables, read them in one place, never print or commit them.
• Follow next_page in a loop (a generator is neat) so you do not miss data.

Next: Reliable API Calls: Timeouts, Retries, Pagination and LLM APIs makes these calls resilient with retries and backoff, sessions and saved results, and then calls the chat API like a real LLM.