Provider-Neutral LLM Clients

Every LLM feature talks to a provider over the network. If that conversation is written straight into your business code, three things hurt: you cannot test without a key and a connection, an SDK change reaches every file, and switching provider means a rewrite. In this tutorial you build a small assistant called EvidenceDesk for the fictional company Kestrel Cycles. This page builds its foundation: a small, provider-neutral client.

You already know HTTP, JSON, classes, pytest and Pydantic from Level 0 (Git, APIs and Backend for AI: Beginner, Python for AI: From Zero to Your First AI App). Here you use them to put a clean boundary between your application and any provider.

What you will learn

• How one request travels: application input, adapter, provider request, response, and the error path.
• How to keep business rules apart from provider code, and how to define a tiny LLMClient contract with typing.Protocol and frozen dataclasses.
• How to write a deterministic FakeClient that returns recorded responses, so every test runs offline.
• What an HTTP request and response to a model API contain, and how adapters for several providers (and a local runtime) map them onto your contract.
• How to prove the contract: the same tests pass against two clients whose wire formats differ.

How to use this tutorial

The 22 pages

The tutorial has 10 Parts of two pages each, then a practice lab and a capstone.

Setup

You need Python 3.12, a virtual environment, pytest and pydantic (used from the next page on). Internet is needed once, for pip. Create the project folder:

# not run here: installs packages (needs internet once)
python3.12 -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
python -m pip install "pytest>=9.0.3" pydantic
mkdir -p evidencedesk fixtures tests
touch evidencedesk/__init__.py

The offline guarantee

Every lab runs with a fake model, recorded fixtures and local data. A missing API key, model download, GPU or internet connection never stops you. Real providers appear only as optional adapters: their code is marked # not run here, has no output, and the facts about them sit in a Provider note dated 2026-10-12. Providers change often, so each note ends by sending you to the current documentation.

How the project grows

You will build one package, evidencedesk/, one small module per page. After this page it looks like this, and every later page adds files next to these:

evidencedesk/
  evidencedesk/__init__.py    (empty)
  evidencedesk/llm.py         types, LLMClient, FakeClient     (this page)
  fixtures/responses.json     recorded responses               (this page)
  tests/test_llm.py           contract tests                   (this page)
  corpus/                     13 synthetic Kestrel Cycles help-centre documents (used from Part 2 on)

Later pages add configuration, context assembly, streaming, embeddings, an index, RAG, a tool, evals, cost controls, security tests and packaging. The corpus is synthetic: no real company or customer data.

An honest note about the fake client

The fake client is a stand-in so that tests are repeatable. It returns text that a person wrote. Real models are less predictable, which is exactly why you test. Fake output shows that your code handles a response; it never shows what a real model would answer.

Map the request lifecycle

Follow one question from a Kestrel Cycles customer through the system. Four stages matter, and one extra path for failure:

user question
    |
    v
APPLICATION   validate input, choose instructions, check the result
    |  LLMRequest   (our type)
    v
ADAPTER       map our request to the provider's request, make the call, map the answer back
    |  provider request: HTTP POST with headers and a JSON body
    v
PROVIDER      runs the model
    |  provider response: status code, headers, JSON body, usage numbers
    v
ADAPTER   --- success --->  LLMResponse (our type)  ---> APPLICATION ---> user
    |
    +------- failure ---->  an error the application can understand (next page)

The application input is what your own code knows. The adapter is the only code that knows a provider. The error path matters as much as the success path: networks time out, quotas run out, keys expire. The next page, Adapters, Configuration, and Error Handling, teaches which failures to retry.

Separate application logic from provider code

Ask of every line: "would this line change if we changed provider?" If yes, it belongs in an adapter. If no, it belongs in the application.

A quick test of the boundary: if you could delete every provider import and all your business-rule tests still passed, the line is in the right place. The tests below run the feature with no provider installed at all.

Design a minimal LLMClient interface

The contract is deliberately small: one method, complete(request) -> LLMResponse. It uses four frozen dataclasses: Message (a role and text), LLMRequest (messages, model name, a token limit), Usage (input and output token counts) and LLMResponse (text, model, usage, stop reason). LLMClient is a typing.Protocol, so an adapter only needs a matching complete method; it does not inherit from anything. frozen=True makes the values immutable and comparable, which makes tests simple.

The file also holds the FakeClient (next section) and one tiny feature, answer_question, which is application logic: it rejects empty input, builds the request and marks truncated answers. It imports no provider. Create the files:

# file: evidencedesk/llm.py
"""Provider-neutral LLM contract: types, the client protocol, a deterministic fake, one tiny feature."""
from __future__ import annotations

import json
from dataclasses import dataclass
from pathlib import Path
from typing import Protocol, runtime_checkable

FIXTURES = Path(__file__).resolve().parent.parent / "fixtures" / "responses.json"
SYSTEM = "Answer only from the Kestrel Cycles help centre. Be brief."


@dataclass(frozen=True)
class Message:
    role: str  # "system", "user" or "assistant"
    content: str


@dataclass(frozen=True)
class LLMRequest:
    messages: tuple[Message, ...]
    model: str = "fake-model"
    max_tokens: int = 256


@dataclass(frozen=True)
class Usage:
    input_tokens: int = 0
    output_tokens: int = 0

    @property
    def total_tokens(self) -> int:
        return self.input_tokens + self.output_tokens


@dataclass(frozen=True)
class LLMResponse:
    text: str
    model: str
    usage: Usage = Usage()
    stop_reason: str = "end"  # our own words: "end", "length" or "other"


@runtime_checkable
class LLMClient(Protocol):
    def complete(self, request: LLMRequest) -> LLMResponse: ...


class FixtureMissing(LookupError):
    """The fake has no recorded response for this input."""


def last_user_text(request: LLMRequest) -> str:
    return next(m.content for m in reversed(request.messages) if m.role == "user").strip()


class FakeClient:
    """Returns recorded responses selected by the last user message. A stand-in, not a model."""

    def __init__(self, fixtures: dict[str, dict] | None = None) -> None:
        self.fixtures = fixtures if fixtures is not None else json.loads(FIXTURES.read_text(encoding="utf-8"))
        self.calls: list[LLMRequest] = []

    def complete(self, request: LLMRequest) -> LLMResponse:
        self.calls.append(request)
        key = last_user_text(request)
        if key not in self.fixtures:
            raise FixtureMissing(f"no recorded response for {key!r}")
        rec = self.fixtures[key]
        usage = Usage(**rec["usage"])
        return LLMResponse(rec["text"], request.model, usage, rec.get("stop_reason", "end"))


def answer_question(client: LLMClient, question: str, system: str = SYSTEM) -> str:
    """Application logic: it knows our rules, not any provider."""
    question = question.strip()
    if not question:
        raise ValueError("empty question")
    request = LLMRequest((Message("system", system), Message("user", question)))
    response = client.complete(request)
    return response.text.strip() + (" [cut off]" if response.stop_reason == "length" else "")

Notice what is missing on purpose. There is no temperature: support depends on the model, and even a temperature of 0 does not guarantee identical output from a real model. Streaming, tools and structured output arrive on later pages. stop_reason uses our own three words, so no provider's words leak into the application. @runtime_checkable allows isinstance(x, LLMClient), but that only checks that a complete method exists, not its signature.

Provider note (as of 2026-10-12): an adapter may have to ignore a field of your request. For example, the Anthropic Python SDK v1 removed temperature, top_p and top_k, and on newer Claude models a non-default value returns HTTP 400. Whether a field is accepted depends on the model, so check the current documentation.

Implement a deterministic fake client

The fake reads recorded responses from a small JSON file, keyed by the exact test input (the last user message). Texts and numbers are written by hand; they are not measurements. Create the fixture file:

# file: fixtures/responses.json
{
  "How long do I have to return a new bike?": {
    "text": "You can return a new, unridden bike within 30 days of delivery for a full refund.",
    "usage": {"input_tokens": 41, "output_tokens": 19},
    "stop_reason": "end"
  },
  "How long does standard delivery take?": {
    "text": "Standard delivery takes 4 to 7 business days. Express delivery takes 2 business days.",
    "usage": {"input_tokens": 38, "output_tokens": 20},
    "stop_reason": "end"
  },
  "How should I store the e-bike battery for a long time?": {
    "text": "For storage longer than 30 days, keep the battery at 40 to 60 percent charge in a dry place and",
    "usage": {"input_tokens": 45, "output_tokens": 20},
    "stop_reason": "length"
  }
}

The block below asks one recorded question twice (to show that nothing changes), then a recorded question whose answer was cut off, then an unknown question:

from evidencedesk.llm import FakeClient, LLMRequest, Message, answer_question

fake = FakeClient()
request = LLMRequest((Message("user", "How long does standard delivery take?"),))
response = fake.complete(request)
print(response.text)
print(response.model, response.usage, response.stop_reason)
print(fake.complete(request) == response)
print(answer_question(fake, "How should I store the e-bike battery for a long time?"))
try:
    answer_question(fake, "Can I pay in instalments?")
except LookupError as error:
    print(type(error).__name__, "-", error)
print(len(fake.calls), "calls recorded")

Standard delivery takes 4 to 7 business days. Express delivery takes 2 business days.
fake-model Usage(input_tokens=38, output_tokens=20) end
True
For storage longer than 30 days, keep the battery at 40 to 60 percent charge in a dry place and [cut off]
FixtureMissing - no recorded response for 'Can I pay in instalments?'
4 calls recorded

Notice four things. Asking twice gives an equal result. The feature, not the fake, added [cut off] for the length stop reason, so you can test that rule without a real truncated answer. An unknown question raises FixtureMissing instead of inventing an answer, so a missing fixture is a loud failure. And calls records every request, so tests can check what the application sent.

Inspect the HTTP request and response

Under every SDK is an HTTP exchange. The request has a method and URL, headers (at least Content-Type: application/json, plus a credential header whose name depends on the provider) and a JSON body with the model name and the messages. The response has a status code, headers and a JSON body with the generated text, a finish indicator and usage metadata. The block below builds a request for a local runtime's chat endpoint with urllib.request.Request but never sends it, so it runs anywhere:

import json
import urllib.request

from evidencedesk.llm import LLMRequest, Message

ours = LLMRequest((Message("system", "Be brief."), Message("user", "How long does standard delivery take?")), model="<local-model>")
body = {"model": ours.model, "messages": [{"role": m.role, "content": m.content} for m in ours.messages], "stream": False}
http = urllib.request.Request(
    "http://localhost:11434/api/chat",
    data=json.dumps(body).encode("utf-8"),
    headers={"Content-Type": "application/json"},
    method="POST",
)  # built, never sent: no urlopen call
print(http.get_method(), http.full_url)
print(dict(http.header_items()))
print(json.dumps(json.loads(http.data), indent=2))

POST http://localhost:11434/api/chat
{'Content-type': 'application/json'}
{
  "model": "<local-model>",
  "messages": [
    {
      "role": "system",
      "content": "Be brief."
    },
    {
      "role": "user",
      "content": "How long does standard delivery take?"
    }
  ],
  "stream": false
}

Notice that urllib normalises the header name to Content-type; HTTP header names are not case-sensitive. The body is only our Message list renamed into dictionaries. A provider response for this call looks like the hand-written illustration below. It is shaped after the local runtime's documented fields; the values are invented and nothing was received from a server:

HTTP/1.1 200 OK
Content-Type: application/json

{"model": "<local-model>", "message": {"role": "assistant", "content": "Standard delivery takes 4 to 7 business days."},
 "done": true, "prompt_eval_count": 41, "eval_count": 19}

An adapter reads three things from it: the text (message.content), the usage (prompt_eval_count and eval_count) and whether generation finished (done). It also reads the status code first, because a non-200 status means there is no answer to read. The next block does that mapping with the dictionary above and one simulated 429 status:

from evidencedesk.llm import LLMResponse, Usage

raw = {"model": "<local-model>", "message": {"role": "assistant", "content": "Standard delivery takes 4 to 7 business days."},
       "done": True, "prompt_eval_count": 41, "eval_count": 19}


def from_chat_json(raw: dict, status: int = 200) -> LLMResponse:
    if status != 200:
        raise RuntimeError(f"HTTP {status}")
    usage = Usage(raw["prompt_eval_count"], raw["eval_count"])
    return LLMResponse(raw["message"]["content"], raw["model"], usage, "end" if raw["done"] else "other")


response = from_chat_json(raw)
print(response.text)
print(response.usage, response.usage.total_tokens)
try:
    from_chat_json({}, status=429)
except RuntimeError as error:
    print("error path:", error)

Standard delivery takes 4 to 7 business days.
Usage(input_tokens=41, output_tokens=19) 60
error path: HTTP 429

Notice the shape of an adapter's work: status check, then field mapping, then our types. Broadly, 400, 401, 403 and 404 mean the request or the access is wrong (do not retry), 429 means a rate limit or an exhausted quota (retry only if it is a rate limit), and 500 and above mean server trouble (retry with a bounded backoff). A timeout is different again: you got no status at all, so you do not know whether the provider did the work. Always set a timeout on the call (the local adapter below uses 60 s, our own choice), treat it as retryable only with a bounded number of attempts, and never retry without a limit. The next page builds the real classification.

Provider note (as of 2026-10-12): usage numbers sit in different places: usage.input_tokens and output_tokens (OpenAI Responses, Anthropic Messages), usageMetadata with names such as candidatesTokenCount (Gemini generateContent), total_input_tokens and similar (Gemini Interactions), prompt_eval_count and eval_count (Ollama REST). Rate-limit headers: OpenAI sends x-ratelimit-* and sometimes Retry-After; Anthropic sends retry-after, anthropic-ratelimit-* and request-id; the Gemini rate-limit page describes none. Check the current documentation.

Add a first provider adapter

An adapter is a class with the complete method that translates in both directions. It does three jobs: map the LLMRequest to the provider's request, call the provider, map the provider's answer to an LLMResponse. A fourth job, mapping failures, comes on the next page. The sketch below uses the SDK of the first provider. You cannot run it without an account, so it has no output; read it to see how little of it is business logic: none.

# not run here: needs the provider SDK, an API key and internet access
from openai import OpenAI

from evidencedesk.llm import LLMRequest, LLMResponse, Usage


class OpenAIAdapter:
    def __init__(self, model: str) -> None:
        self.client, self.model = OpenAI(), model  # reads OPENAI_API_KEY

    def complete(self, request: LLMRequest) -> LLMResponse:
        system = "\n".join(m.content for m in request.messages if m.role == "system")
        user = "\n".join(m.content for m in request.messages if m.role == "user")
        r = self.client.responses.create(model=self.model, instructions=system, input=user)
        usage = Usage(r.usage.input_tokens, r.usage.output_tokens)
        return LLMResponse(r.output_text, self.model, usage, "end" if r.status == "completed" else "other")

Notice that the system text goes into instructions, the answer is output_text, and the usage fields are renamed into our Usage. The status maps to our two words. Our max_tokens is not passed in this sketch; whether and how to map it is a decision to take from the current documentation. When you run it with a real key, the answer text comes from the model, so do not assert on its exact wording. No output is printed here because none was produced.

Provider note (as of 2026-10-12): OpenAI's Python package is openai (pip install openai; the latest on PyPI on that date was 3.29.0). Its SDK page calls the Responses API the primary API; the older Chat Completions API stays supported. The call is client.responses.create(model=..., input=...), with system-style guidance in instructions=, text in response.output_text, and response.status values such as completed and incomplete. The docs showed the example model ID gpt-6-astra; treat it as an example only and copy a current ID from the provider's models page. VERIFY — checked 2026-10-12. Check the current documentation.

Add a second provider adapter

The second provider organises the same information differently, and that is the point of an adapter. Here the system text is a top-level field and not a message, the examples always set a token limit, the answer is a list of content blocks, and the stop reason has its own words. The application code and your tests do not change:

# not run here: needs the provider SDK, an API key and internet access
import anthropic

from evidencedesk.llm import LLMRequest, LLMResponse, Usage


class AnthropicAdapter:
    def __init__(self, model: str) -> None:
        self.client, self.model = anthropic.Anthropic(), model  # reads ANTHROPIC_API_KEY

    def complete(self, request: LLMRequest) -> LLMResponse:
        system = "\n".join(m.content for m in request.messages if m.role == "system")
        chat = [{"role": m.role, "content": m.content} for m in request.messages if m.role != "system"]
        m = self.client.messages.create(model=self.model, max_tokens=request.max_tokens, system=system, messages=chat)
        text = "".join(b.text for b in m.content if b.type == "text")
        return LLMResponse(text, self.model, Usage(m.usage.input_tokens, m.usage.output_tokens),
                           {"end_turn": "end", "max_tokens": "length"}.get(m.stop_reason, "other"))

Notice the two translations that the first adapter did not need: removing the system message from the list, and joining text blocks. The stop reason goes through a small dictionary into our vocabulary; unknown values become other, never an exception.

Provider note (as of 2026-10-12): Anthropic's Python package is anthropic (pip install anthropic; latest on PyPI on that date 1.13.0). The call is client.messages.create(model=..., max_tokens=..., messages=[...]); max_tokens is shown in the examples, and the system prompt is the top-level system field, not a message. The answer is message.content, a list of blocks; text blocks have type text. Stop reasons include end_turn and max_tokens. The docs showed the example model ID claude-opus-5-5, an example only. VERIFY — checked 2026-10-12. Check the current documentation.

Compare Google's SDK and Local Runtimes

Two more adapters complete the picture. The Google one uses the Google Gen AI SDK. The local one skips an SDK completely and sends the HTTP request you built above to a runtime on your own machine, whose documented example needs no API key and no internet:

# not run here: needs the provider SDK, an API key and internet access
from google import genai

from evidencedesk.llm import LLMRequest, LLMResponse


class GeminiAdapter:
    def __init__(self, model: str) -> None:
        self.client, self.model = genai.Client(), model  # reads GEMINI_API_KEY

    def complete(self, request: LLMRequest) -> LLMResponse:
        system = "\n".join(m.content for m in request.messages if m.role == "system")
        user = "\n".join(m.content for m in request.messages if m.role == "user")
        i = self.client.interactions.create(model=self.model, system_instruction=system, input=user)
        return LLMResponse(i.output_text, self.model)  # usage: attribute path not confirmed, check the docs

Notice that the usage mapping is missing on purpose: the exact attribute path is not confirmed, so the sketch returns the default Usage() (zeros) and says so, rather than guessing. A real adapter must not hide a gap like this.

# not run here: needs a local runtime listening on localhost:11434 and a pulled model
import json, urllib.request

from evidencedesk.llm import LLMRequest, LLMResponse, Usage


class LocalAdapter:
    def __init__(self, model: str, url: str = "http://localhost:11434/api/chat") -> None:
        self.model, self.url = model, url

    def complete(self, request: LLMRequest) -> LLMResponse:
        messages = [{"role": m.role, "content": m.content} for m in request.messages]
        body = json.dumps({"model": self.model, "messages": messages, "stream": False}).encode("utf-8")
        http = urllib.request.Request(self.url, body, {"Content-Type": "application/json"})
        raw = json.load(urllib.request.urlopen(http, timeout=60))  # the 60 s is our choice, not a documented default
        usage = Usage(raw["prompt_eval_count"], raw["eval_count"])
        return LLMResponse(raw["message"]["content"], self.model, usage, "end" if raw["done"] else "other")

Notice that this adapter is the request block plus the mapping block from earlier: mechanism before library.

Provider note (as of 2026-10-12): Gemini has two API paths. Google's documentation leads with the Interactions API (client.interactions.create(model=..., input=...), text in interaction.output_text, system text in system_instruction=) and calls it recommended for new projects. The older generateContent path (client.models.generate_content) is called legacy but fully supported, and the SDK README still shows it first. The package is google-genai (latest 2.29.0 on that date). Example model ID in the docs: gemini-3.8-flash. The Python attribute names for usage on both paths were not confirmed. Local runtime: Ollama is a separate application; its API listens on http://localhost:11434; POST /api/chat with model, messages and stream returns message.content, done, prompt_eval_count and eval_count; the Python package ollama offers chat(...). A model must be pulled by name first (example in the docs: gemma4:e2b). The Python client's default timeout and retries are not documented, so assume none. VERIFY — checked 2026-10-12. Check the current documentation.

Whatever the provider, an adapter must find the text (maybe in several parts), the token counts (if reported), why generation stopped (translated into end, length or other), and the failure (translated into one of our errors, next page). Features differ too (streaming, structured output, tools, embeddings), so never assume parity; the next page adds a capability matrix.

Exercise: swap providers without rewriting the feature

Real adapters cannot run here, so prove the idea offline. ShapeBFake below imitates a provider with another wire format: its request body uses prompt items with who and text, and its reply has parts, tokens and a finish value (STOP or CUT). The function from_shape_b plays the adapter. One set of tests then runs against both clients. Create the file and run it:

# file: tests/test_llm.py
import dataclasses

import pytest

from evidencedesk.llm import (FakeClient, FixtureMissing, LLMClient, LLMRequest, LLMResponse, Message, Usage,
                              answer_question)

Q_RETURN = "How long do I have to return a new bike?"
Q_BATTERY = "How should I store the e-bike battery for a long time?"


def to_shape_b(request: LLMRequest) -> dict:
    """What a different provider's HTTP body could look like."""
    return {"prompt": [{"who": m.role, "text": m.content} for m in request.messages], "limit": request.max_tokens}


def from_shape_b(raw: dict, model: str) -> LLMResponse:
    """The adapter step: provider shape in, our LLMResponse out."""
    text = "".join(part["text"] for part in raw["parts"])
    usage = Usage(raw["tokens"]["in"], raw["tokens"]["out"])
    return LLMResponse(text, model, usage, {"STOP": "end", "CUT": "length"}.get(raw["finish"], "other"))


class ShapeBFake:
    """Imitates a provider with another wire format; the adapter maps it back."""

    def __init__(self) -> None:
        self.inner = FakeClient()

    def complete(self, request: LLMRequest) -> LLMResponse:
        body = to_shape_b(request)  # the provider only ever sees its own format
        key = [item["text"] for item in body["prompt"] if item["who"] == "user"][-1].strip()
        if key not in self.inner.fixtures:
            raise FixtureMissing(key)
        rec = self.inner.fixtures[key]
        cut = len(rec["text"]) // 2
        raw = {"parts": [{"text": rec["text"][:cut]}, {"text": rec["text"][cut:]}],
               "tokens": {"in": rec["usage"]["input_tokens"], "out": rec["usage"]["output_tokens"]},
               "finish": "CUT" if rec.get("stop_reason") == "length" else "STOP"}
        return from_shape_b(raw, request.model)


@pytest.fixture(params=[FakeClient, ShapeBFake], ids=["fake", "shape-b"])
def client(request):
    return request.param()


def test_same_feature_same_answer(client):
    assert isinstance(client, LLMClient)
    assert answer_question(client, Q_RETURN) == "You can return a new, unridden bike within 30 days of delivery for a full refund."


def test_truncation_rule_lives_in_the_feature(client):
    assert answer_question(client, Q_BATTERY).endswith("and [cut off]")


def test_unknown_input_is_an_error_not_a_guess(client):
    with pytest.raises(FixtureMissing):
        answer_question(client, "Can I pay in instalments?")


def test_empty_question_rejected_before_any_call():
    fake = FakeClient()
    with pytest.raises(ValueError):
        answer_question(fake, "   ")
    assert fake.calls == []


def test_fake_is_deterministic():
    request = LLMRequest((Message("user", Q_RETURN),))
    assert FakeClient().complete(request) == FakeClient().complete(request)


def test_usage_and_frozen_types():
    response = FakeClient().complete(LLMRequest((Message("user", Q_RETURN),)))
    assert response.usage.total_tokens == 60
    with pytest.raises(dataclasses.FrozenInstanceError):
        response.text = "changed"

python -m pytest -q tests/test_llm.py

.........                                                                [100%]
9 passed in 0.09s

Notice that every contract test ran twice, once per client (the two fake and shape-b parameters), and answer_question was never edited. That is what "swap providers without rewriting the feature" means: only the object passed in changes.

What these tests cannot show: they prove that your code respects the contract, not that a real adapter maps a real provider correctly. A fake you wrote will agree with the adapter you wrote. Before relying on an adapter, run it once against the real service (optional, and outside this tutorial's offline path) and keep a small, sanitised recorded response as a fixture.

Where this shows up at work

Teams change providers for cost, quality, data-location rules or an outage. Teams that wrote against a narrow interface change one adapter and re-run their tests; teams that scattered SDK calls through the code change dozens of files. The same boundary lets continuous integration run without secrets.

Common mistakes

• Importing the provider SDK in business code. Every test then needs a key and every file is tied to one vendor. Keep imports inside adapters.
• Passing the raw provider dictionary upward. Callers then depend on its shape. Convert at the adapter edge.
• A fake that guesses. A fake that returns a default for unknown input hides missing fixtures and makes tests pass for the wrong reason. Raise an error.
• Treating fake output as model behaviour. A fixture shows how your code handles a reply, not what a real model answers; a real model at temperature 0 can still vary between runs.

Try it yourself

1. Easy. Add a fixture for the question "Where do you deliver?" (use the shipping document in corpus/ for the text) and show that answer_question returns it without any change to llm.py.
2. Medium. Revisit HTTP from Git, APIs and Backend for AI: Beginner. Build, but do not send, a urllib.request.Request whose JSON body is to_shape_b(request), and assert the method, the content type and that the body decodes back to the same dictionary.
3. Hard. Write a RecordingClient that wraps any LLMClient, stores each (request, response) pair in a list, and is itself a valid LLMClient. Test it by wrapping ShapeBFake.

Answers

One test file holds all three solutions; it imports names from the file you already wrote.

# file: tests/test_exercises.py
import json
import urllib.request

from test_llm import Q_RETURN, ShapeBFake, to_shape_b

from evidencedesk.llm import FakeClient, LLMClient, LLMRequest, Message, answer_question


def test_easy_new_fixture_without_touching_the_feature():
    question = "Where do you deliver?"
    record = {"text": "We deliver to 12 countries.", "usage": {"input_tokens": 20, "output_tokens": 8}}
    assert answer_question(FakeClient({question: record}), question) == "We deliver to 12 countries."


def test_medium_http_body_round_trip():
    request = LLMRequest((Message("user", "Hi"),), max_tokens=50)
    http = urllib.request.Request("https://llm.example.test/v1/respond", data=json.dumps(to_shape_b(request)).encode(),
                                  headers={"Content-Type": "application/json"}, method="POST")
    assert http.get_header("Content-type") == "application/json"
    assert json.loads(http.data) == {"prompt": [{"who": "user", "text": "Hi"}], "limit": 50}


class RecordingClient:
    def __init__(self, inner: LLMClient) -> None:
        self.inner, self.log = inner, []

    def complete(self, request):
        response = self.inner.complete(request)
        self.log.append((request, response))
        return response


def test_hard_a_wrapper_is_a_client_too():
    client = RecordingClient(ShapeBFake())
    assert isinstance(client, LLMClient)
    answer_question(client, Q_RETURN)
    assert len(client.log) == 1 and client.log[0][1].usage.total_tokens == 60

python -m pytest -q

............                                                             [100%]
12 passed in 0.11s

Pass checklist:

• All tests pass, and llm.py was not edited for the Easy task.
• The Medium test never opens a connection (no urlopen).
• The Hard wrapper has a complete(request) method, returns the inner response unchanged, and passes isinstance(client, LLMClient).

Summary

• A request travels application, adapter, provider and back; the error path is part of the design.
• Keep business rules in the application and everything provider-specific in adapters.
• Adapters differ in where they put system text, answers, usage and stop reasons, and providers do not offer equal features; each adapter hides the differences.
• The contract is one method, complete(request) -> LLMResponse, with frozen dataclasses and a Protocol.
• A deterministic fake with recorded fixtures makes tests offline and repeatable, but it is a stand-in: it never shows real model behaviour.

Next: Adapters, Configuration, and Error Handling. You will keep model names and settings outside the code, read secrets safely, and classify errors into retryable and not retryable.