Practice Lab

Aiko in Osaka is asked to show that she can take an AI feature from a design to a release that a team can trust. A graded run does: fixed cases, a score, and a short list of gates that fail the whole job however many points you collect. This page is that run for the Advanced tutorial. You build six small modules for a production-minded knowledge service, and a grader scores them out of 100. (An illustrative case.)

Everything is offline. The model is a stand-in, the embedder is a toy that matches letters, and the agent is scripted; real models are less predictable, which is exactly why you test. This page adds a folder kit/advlab/ and the test file kit/tests/test_advlab.py, and changes nothing that exists. Honest limit up front: the grader runs your code in its own process with no sandbox, so run only code you wrote.

What you will learn

• How the lab is built: six challenges, 100 points, a pass mark of 75, and five critical gates that outrank the score.
• How each Advanced Part becomes a fixture, a contract and an automated pass criterion, with starter files, hints and a model solution.
• How to prove that a gate works, by building one flawed submission per gate that otherwise scores 100.
• What a fixture grader cannot show, and what evidence to submit: report, trace sample, threat model, deployment instructions, limitations.

How the lab works

The lab is called "Production RAG and Agent Verification". You write six Python modules in a folder submission/. The grader imports them and feeds them fixtures: a labelled mini-corpus, MCP cases, agent scenarios, an evaluation dataset with judge labels, synthetic traces and usage records, tenant documents and release plans. You may import the whole evidencedesk package; the starter docstrings are the contracts.

Pass: at least 75 points, and all five critical gates green, and a complete submission. A missing key or connection never blocks a pass: the grader switches key variables and sockets off while it runs. Required submission: source, tests, an evaluation report, a trace sample, a threat model, deployment instructions and known limitations. The grader checks that each exists, is not a placeholder, and that the two JSON files pass its own rules; it cannot judge whether they are good. Start from the starter; every stub raises NotImplementedError and its docstring is the contract (names, types, return shapes, codes).

cp -r advlab/starter submission

Grade the untouched starter (the first six lines and the last). The score is 0, and that is the right answer. No gate is red, because nothing is written that could leak or run; an untouched TODO plan counts as "not written yet", and the submission list names what is missing.

python -m advlab.grade submission | sed -n '1,6p;$p'

1. Hybrid retrieval benchmark      0/20
2. MCP integration                 0/15
3. Agent control                   0/15
4. Evaluation gate                 0/20
5. LLMOps and security             0/20
6. Deployment and recovery         0/10
SCORE 0/100 | critical failures 0 | RESULT: FAIL

Fixtures and the grader

Fixtures are public files in advlab/fixtures/; read them first, they are the specification. Three rows show the idea: a query with graded relevance, an agent probe, and a release-gate case with its expected verdict. Note what the fixtures do not contain: no key-, card- or address-shaped text. Redaction cases hold placeholders such as {EMAIL}, and advlab/bait.py builds the values at run time, so the release scan of the kit stays clean.

from advlab.grade import rows, table

q = rows("qrels")[5]
print("qrels", q["id"], q["query"], q["grades"])
p = rows("agent_probes")[0]
print("probe", p["id"], p["why"])
print("  proposals", p["proposals"])
g = table("gate_cases")["cases"][3]
print("gate", g["id"], g["verdict"], g["reasons"])

qrels q06 when will I get my money refunded {'kc-refunds': 2, 'kc-returns-v3': 1}
probe x1 an urgent flag must not skip the approval step
  proposals [{'tool': 'send_email', 'arguments': {'to': 'customer', 'body': 'hi', 'urgent': True}}]
gate g4 block ['floor:answer_correct', 'regress:answer_correct']

The grader (python -m advlab.grade <folder> [--verbose] [--no-tests]) is one file. Each check is a function registered with a challenge, a name and points; a failed assertion is its message. A broken file costs only its own challenge. Two honest limits. First, fixtures only: a solution can pass by handling exactly these cases, so add your own. Second, points are all or nothing per check, so read the message before you change code. The excerpt is the critical gate for isolation: one shared search object that every user queries in turn. (The points checks use a fresh object per call.)

# excerpt of kit/advlab/grade.py
def critical_checks(ctx: Ctx) -> list[tuple[str, bool, str]]:
    out = []
    # 1. cross-tenant: ONE shared instance, every principal asks every query in turn (a cache or an index shared between users shows up here)
    problems = []
    if "llmops" in ctx.mods:
        spec, principals = tenant_world()
        try:
            shared = ctx.m("llmops").TenantSearch(json.loads(json.dumps(spec["docs"])))
            for q in spec["queries"] * 2:
                for name in ("nadia", "omar", "guest", "omar", "nadia"):
                    got = shared.search(dict(principals[name]), q["query"], 3)
                    if foreign := sorted(set(got) - allowed_ids(spec, principals[name])):
                        problems.append(f"{name} saw {foreign[0]}")
        except Exception:  # a crash exposes nothing; the points checks already cost its challenge
            pass
    out.append((CRITICAL[0], not problems, "; ".join(dict.fromkeys(problems))[:90]))
    ...

Challenge 1: Hybrid retrieval benchmark (20 points)

Task. In bench.py write rank_keyword, rank_dense, fuse, ranking_metrics, run_benchmark and recommend. Dense, keyword and hybrid run on one labelled corpus (18 short synthetic documents, 14 queries with grade 2 = answers and 1 = related). Metrics, stated because libraries differ: recall and MRR count grade 1 or more as relevant; nDCG uses gain 2^grade - 1 and discount log2(rank + 1). Fusion is reciprocal rank fusion with k = 60, the constant of the original paper. The recommendation must state its rule, quote its numbers and name the number of queries. Fixtures: corpus_mini.jsonl, qrels.jsonl, retrieval_checks.json (metric, fusion and recommendation cases).

Hints. Hybrid Retrieval and Error Analysis (BM25, dense, fusion); Reranking and Retrieval Evaluation (recall, MRR, nDCG, intervals); Beginner Local Indexes and Retrieval Quality. A good solution orders ties by something stable, never by set order, and decides before it looks: "hybrid must beat the best single method by 0.02 nDCG, because it costs two indexes and a fusion step". Look at what the kit's reference run shows per question, as the rank of the first relevant document (a dash means not found):

import json, sys
sys.path.insert(0, "advlab/reference")
import bench

docs = {r["id"]: r["text"] for r in map(json.loads, open("advlab/fixtures/corpus_mini.jsonl"))}


def first_rank(ranked, grades):
    return next((n for n, d in enumerate(ranked, 1) if grades.get(d, 0) >= 1), None)


print("id  query                              kw  dense hybrid")
for q in map(json.loads, open("advlab/fixtures/qrels.jsonl")):
    kw, de = bench.rank_keyword(q["query"], docs), bench.rank_dense(q["query"], docs)
    ranks = [first_rank(r, q["grades"]) for r in (kw, de, bench.fuse([kw, de]))]
    print(q["id"], f"{q['query'][:34]:<34}", f"{ranks[0] or '-':>3}", f"{ranks[1] or '-':>5}", f"{ranks[2] or '-':>6}")

id  query                              kw  dense hybrid
q01 RD2-24                               1     1      1
q02 what does E-17 mean                  1     1      1
q03 is my bike part of recall RD3-11     1     1      1
q04 how long to send the bike back       2     1      1
q05 returning a bike after delivery      2     1      1
q06 when will I get my money refunded    -     1      1
q07 charged the battery too hot          1     1      1
q08 how to store the battery for winte   1     1      1
q09 how many days does shipping take     1     2      1
q10 warranty on the electrical motor     1     1      1
q11 brake lever problem stop riding      1     1      1
q12 my bike was stolen                   1     2      2
q13 pressure for the tyres               1     1      1
q14 can I cancel my order                1     1      1

Keyword search finds nothing for "money refunded" because the word is not in the page, and the toy embedder ranks the stolen-bike page second where keyword is first. Hybrid recovers most of both, not all. Now the recommendation the reference prints:

PYTHONPATH=.:advlab/reference python advlab/reference/demo.py | sed -n 1,9p

method   recall@5    mrr  ndcg@5
keyword     0.821  0.857   0.829
dense       0.893  0.929   0.893
hybrid      0.893  0.964   0.922
queries: 14
recommendation: hybrid
  keyword ndcg 0.829 recall 0.821 mrr 0.857
  dense ndcg 0.893 recall 0.893 mrr 0.929
  hybrid ndcg 0.922 recall 0.893 mrr 0.964

Hybrid wins by 0.029 nDCG, which clears the 0.02 margin chosen in advance, and the gain is in MRR (the right page comes first more often), not in recall. With 14 queries that is a few questions moving one rank; a margin of 0.03 would flip the recommendation to dense. That is the honest reading, and recommend must say so in its caveats.

Challenge 2: MCP integration (15 points)

Task. In mcpint.py write build_server(corpus_dir), an in-memory MCP server with two read-only tools (lookup_document, search_help), and call(server, name, arguments, timeout, max_bytes), a client that returns {ok, code, data} and never raises. Fixtures: mcp_cases.jsonl (eight calls). The grader also builds a rogue server that accepts anything and records what reaches it, and uses two hostile servers from the kit: one with a 5,000-character reply, one that hangs.

Tool note (as of 2026-10-12): the lab was run with the mcp Python SDK 2.3.0 (the version the kit expects), where the server class is MCPServer and a Client can connect to a server object in memory, which is how the grader tests without a process or a port. The current specification version is 2026-07-28; it has no initialize handshake and no sessions. These change, so check the current documentation and re-pin.

Hints. MCP Architecture and Local Server (tools, schemas, in-memory tests); MCP Security and Interoperability (untrusted results, size caps, timeouts); Beginner Tool Contracts and Execution. A good solution validates twice, because the SDK rejects a wrong type but silently drops an extra field, so only your own check protects a server that accepts anything. It refuses an oversized reply instead of clipping it, treats server text as untrusted data, and keeps paths out of errors. Protocol interoperability is not security.

Challenge 3: Agent control (15 points)

Task. In control.py write run_agent(proposals, tools, policy) and resume(run, approver, decision, tools, policy). Proposals are an iterator of untrusted dicts that may never end. The policy holds an allowlist, a set of tools that need a person, max_steps, the requester and the approvers. Fixtures: agent_scenarios.jsonl (13 runs) and agent_probes.jsonl (8 attacks). The tools the grader passes in record every call: the critical gate reads what they recorded, never what you report.

Hints. Agent Loops, State, and Memory (state, step budgets, stop reasons); Human Approval and Multi-Agent Boundaries (approval queues, self-approval); Beginner Tool Permissions and Failure Handling. A good solution checks the step limit before it reads the next proposal, matches tool names exactly (a space or a capital letter is a different tool), and keeps approval outside the model: the requester is refused first, even when the approver list names them. It updates the run in place so a repeated approval finds the decision made. A scripted proposer cannot show how a real model behaves under pressure.

Challenge 4: Evaluation gate (20 points)

Task. In evalgate.py write dataset_version(cases), judge_report(rows, prompt_sha, calibrated_with) and decide(candidate, baseline, policy). The dataset is versioned by content. A judge is calibrated only with at least 20 labelled pairs, kappa of 0.6 or more against the human labels, and an unchanged prompt; otherwise it is experimental and its metrics inform but never gate. The gate returns go, review or block with sorted reason codes. Fixtures: eval_cases.jsonl, judge_cases.json (four judges: good, tiny sample, always picks "a", stale prompt), gate_cases.json (nine candidates). The thresholds 20 and 0.6 are lab choices.

Hints. Evaluation Design and Judge Calibration (versioned data, agreement, biases); Robust Metrics and Release Gates (floors, no-regress margins, critical cases); Beginner Rubrics, Regression, and Release Gates. A good solution hashes the content, not the file; treats a missing critical case as a failure; refuses to call a run a regression when the dataset changed; and says out loud that kappa from 24 pairs has a wide interval. The judge labels here are synthetic, so they show the logic of calibration, not how good any real judge is.

Challenge 5: LLMOps and security (20 points)

Task. In llmops.py write redact, trace_problems, usage_totals, screen_chunks and the class TenantSearch. Fixtures: redaction_cases.jsonl, trace_cases.jsonl (two good traces, seven defective), usage_cases.json (one replayed record), injection_cases.jsonl (six attacks, six harmless chunks that use words such as "ignore" and "forward"), tenant_cases.json (two tenants plus public documents).

Hints. Structured Logging and Tracing (what a span records and what it must not); Monitoring, Budgets, and Incidents (usage and budgets); Threat Modelling and Injection Testing; Guardrails and Security Regression (redaction, screening, regression suites); Ingestion and Permission-Aware Retrieval (filter before ranking). A good solution filters by tenant before ranking, on every call, and keeps no cache or index that two callers share. It records counts and ids in a trace, never question text, and treats a pattern list for injection as a signal for review; permissions are the control.

Security note (as of 2026-10-12): if your THREAT_MODEL.md cites a public risk list, name the edition. The OWASP Top 10 for LLM Applications 2025 edition was the one on the project's main page, and a newer 2026 edition was listed; check the current documentation and say which one you used.

Challenge 6: Deployment and recovery (10 points)

Task. In recovery.py write plan_problems(plan), a static checker that returns sorted codes for a release plan, and make_app(env), a small app that starts in mock mode and refuses a half-configured live mode. The documented local equivalent of a container is this app plus its smoke tests. Fixtures: plans.jsonl (15 plans covering every code, one a combination) and good_plan_example.json.

Hints. Containerisation and Streaming Deployment (settings from the environment, smoke tests, readiness); Scaling, Resilience, and Rollback (exact previous versions, drain, rollback plans); Release Evidence and Production Readiness (a rollback that someone has verified). A good solution fails at start-up rather than at the first request, keeps secrets in the environment and out of files, and writes a rollback plan with a trigger, an owner, a time budget and a check to run afterwards. A plan checker shows the plan is complete, not that the rollback works; only a drill shows that.

Critical gates

These five gates are worth no points; they decide whether the points count. Each is an automated probe, not a promise: a leak the probe does not trigger still passes.

A grader you have not tried to fool is only an opinion. advlab/bad.py builds a copy of the reference with exactly one flaw per gate. The cross-tenant flaw is the instructive one: a cache keyed by the query alone. The points checks build a fresh search object for every call, so they never see it.

# excerpt of kit/advlab/bad.py
    "cross_tenant": ("llmops.py", '''

_search_once = TenantSearch.search


def _search_cached(self, principal, query, k=3):  # "faster": the same question gets the same answer
    cache = self.__dict__.setdefault("_cache", {})
    if (query, k) not in cache:
        cache[(query, k)] = _search_once(self, principal, query, k)
    return cache[(query, k)]


TenantSearch.search = _search_cached
'''),

Grade the five copies. The key in the third is assembled at run time, so the lab's own files hold no key-shaped text, and bad/ is deleted afterwards because the release check of Release Evidence and Production Readiness would rightly refuse it.

for flaw in cross_tenant tool_exec secret nondeterministic no_rollback; do
  python -m advlab.bad $flaw bad/$flaw > /dev/null
  echo "== $flaw"
  python -m advlab.grade bad/$flaw --no-tests | grep -E 'FAIL|SCORE'
done
rm -rf bad

== cross_tenant
  [FAIL] no cross-tenant exposure   omar saw acme-refunds; guest saw acme-refunds; omar saw acme-sla; guest saw acme-sla
SCORE 100/100 | critical failures 1 | RESULT: FAIL
== tool_exec
  [FAIL] no unauthorised tool run   x1: send_email ran without a valid approval
SCORE 100/100 | critical failures 1 | RESULT: FAIL
== secret
  [FAIL] no credentials in source   1 finding(s): ['local_settings.py:1 provider-style key']
SCORE 100/100 | critical failures 1 | RESULT: FAIL
== nondeterministic
  [FAIL] reproducible offline       the same inputs gave different results in two runs
SCORE 100/100 | critical failures 1 | RESULT: FAIL
== no_rollback
  [FAIL] rollback or safe shutdown  budget_not_credible, no_owner, no_rollback_steps
  [FAIL] rollback_plan.json         budget_not_credible, no_owner, no_rollback_steps
SCORE 100/100 | critical failures 1 | RESULT: FAIL

Each copy fails exactly one gate, and the grader names the case or file. All five score 100 on points, so points alone would have passed them. The nondeterministic copy adds a run id from the clock: every check still passes, but two runs of the same inputs never match. The last copy also fails a submission item, which is right: the plan is a required file.

The reference solution

advlab/reference/ answers all six challenges with thin layers over the kit (BM25 and fusion, the MCP server and guard, the evaluation kit, the security helpers), plus the evidence files, its own nine tests and an example output that a test regenerates. Try the challenges before you read it. Here is its real grade.

python -m advlab.grade advlab/reference

1. Hybrid retrieval benchmark     20/20
2. MCP integration                15/15
3. Agent control                  15/15
4. Evaluation gate                20/20
5. LLMOps and security            20/20
6. Deployment and recovery        10/10
Critical gates
  [ok  ] no cross-tenant exposure
  [ok  ] no unauthorised tool run
  [ok  ] no credentials in source
  [ok  ] reproducible offline
  [ok  ] rollback or safe shutdown
Required submission
  [ok  ] README.md
  [ok  ] EVAL_REPORT.md
  [ok  ] THREAT_MODEL.md
  [ok  ] DEPLOY.md
  [ok  ] LIMITATIONS.md
  [ok  ] trace_sample.json
  [ok  ] rollback_plan.json
  [ok  ] fixture dataset
  [ok  ] tests written              9 test function(s), need 6+
  [ok  ] cross-tenant test          a test that mentions tenants is required
  [ok  ] own tests pass
SCORE 100/100 | critical failures 0 | RESULT: PASS

It scores 100 of 100 and passes. Read that number carefully: the author knew the fixtures, so it shows that the grader and a correct solution agree, not that the solution is good. The retrieval numbers come from 14 queries and a toy embedder, the judge labels are synthetic, the agent is scripted, the MCP server never leaves memory. Look for misses in your own fixtures/.

Optional extensions

One extension needs a download the lab does not assume. It has no output here because none was produced, it is never graded, and a missing package or download never changes your score. A container run (build, start with APP_MODE=mock, call the smoke routes) is a second optional extension. Never copy the toy numbers over: run the same benchmark with the real model and report what it prints.

# not run here: needs the sentence-transformers package and a model download on first use
import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")  # English only


def rank_dense_real(query: str, docs: dict[str, str], k: int = 10) -> list[str]:
    ids = sorted(docs)
    vectors = model.encode([docs[i] for i in ids], normalize_embeddings=True)
    scores = vectors @ model.encode(query, normalize_embeddings=True)
    return [ids[i] for i in np.argsort(-scores, kind="stable")[:k]]  # run the same benchmark; never copy the toy numbers over

Provider note (as of 2026-10-12): sentence-transformers was at version 6.1.0 (released 2026-09-18, Apache-2.0); the loading call is SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2") followed by model.encode([...]). That model is English only (384 dimensions, about 22.7 million parameters) and downloads its weights on first use; for Bangla, multilingual-e5-small was the documented option, with lower quality. Model weights have their own licences. VERIFY — checked 2026-10-12. Check the current documentation.

Submit your work

Fill the template files last, from a real run, then grade without --no-tests so your own tests run. The report holds your benchmark table, recommendation and gate result; the threat model lists threats, the control in code for each and the test that proves it; trace_sample.json holds one request of ids, times and counts, no text; rollback_plan.json follows good_plan_example.json. "All tests pass" is not a limitation.

Where this shows up at work

Lena in Berlin reviews a pull request that adds an agent to a support tool. She does not read the prompt first. She asks the five questions of this grader: can one customer see another's data, can the agent run something nobody approved, is there a secret in the diff, can she reproduce the numbers on her laptop, and what happens when it must be switched off. Teams run such probes as a gate in continuous integration. A candidate who says "I scored 94, my own fixtures found two misses, here is the rollback drill" is more convincing than a perfect score with no evidence.

Common mistakes

• Coding to the fixtures. A solution can special-case the files. Add your own cases and test those.
• Testing isolation with a fresh object. A cache or index shared between users leaks only when the same object serves two people. Test the shared case.
• Letting the model's words gate an action. The approval and the allowlist live in code. A prompt that says "ask first" is not a control.
• A recommendation without its rule. "Hybrid is best" with no margin, no sample size and no caveat is an opinion. State the rule before you look.
• Treating the scan as a guarantee. It finds shapes, not secrets in history or in plain words.

Try it yourself

1. Easy. Revisit Robust Metrics and Release Gates. Copy the starter and add only the reference evalgate.py. Predict the score, the gates and the result before you grade.
2. Medium. Revisit Reranking and Retrieval Evaluation. In a copy of the reference set MIN_GAIN to 0.0. Predict which check fails and what the result is.
3. Hard. Revisit Guardrails and Security Regression. Write leaky_trace.py, which adds either a raw question or an address-shaped note to the trace sample of a copy of the reference. Predict, for each, the score, whether a gate fails and which line names it. Should the first one be a gate?

Answers

Easy. One module worth 20 points, so the score is 20 and the result is FAIL: the pass mark is 75 and the submission files are placeholders. No gate fails, because nothing else is written.

rm -rf submission
cp -r advlab/starter submission
cp advlab/reference/evalgate.py submission/evalgate.py
python -m advlab.grade submission --verbose --no-tests | sed -n '/^4\./p;$p'

4. Evaluation gate                20/20
SCORE 20/100 | critical failures 0 | RESULT: FAIL

Medium. With no margin, hybrid is recommended whenever it is ahead by anything. On the real data that is still hybrid, so only the case "hybrid inside the margin" fails, two points are lost, and the result is still PASS. That is the danger of a pass mark: a real regression hides under it.

cp -r advlab/reference copy
sed -i 's/MIN_GAIN = 0.02/MIN_GAIN = 0.0/' copy/bench.py
python -m advlab.grade copy --verbose --no-tests | grep -E -- '\[--\]|SCORE'

     [--] recommend_choice           0/2  hybrid inside the margin, dense best single: chose hybrid, expected dense
SCORE 98/100 | critical failures 0 | RESULT: PASS

Hard. A raw question in the trace is caught by the submission check (raw_content) and no gate fails; an address-shaped value is caught twice, and the credential and personal-data scan turns it into a gate failure. The script builds the address at run time, so the lab's files stay clean.

# file: leaky_trace.py
"""Add one thing to the trace sample of a copy of the reference, then grade the copy."""
import json
import sys
from pathlib import Path

variant, path = sys.argv[1], Path("copy/trace_sample.json")
trace = json.loads(path.read_text(encoding="utf-8"))
# the address is built here, so this file holds no address-shaped text
extra = {"question": "How do I return my bike?"} if variant == "question" else {"note": "customer " + "nadia.k" + "@" + "mailbox.invalid"}
trace["spans"][1]["attrs"].update(extra)
path.write_text(json.dumps(trace, indent=1) + "\n", encoding="utf-8")

for variant in question email; do
  rm -rf copy && cp -r advlab/reference copy
  python leaky_trace.py $variant
  echo "== $variant"
  python -m advlab.grade copy --no-tests | grep -E 'FAIL|SCORE'
done

== question
  [FAIL] trace_sample.json          raw_content
SCORE 100/100 | critical failures 0 | RESULT: FAIL
== email
  [FAIL] no credentials in source   1 finding(s): ['trace_sample.json:24 email']
  [FAIL] trace_sample.json          pii_in_attrs
SCORE 100/100 | critical failures 1 | RESULT: FAIL

Whether a raw question in a trace should be a gate is a design choice: here the file check fails the submission, but the gate list stays green, a gap a sixth gate could close. The lab is also under test (the sed removes the wall-clock time): the reference, the starter, the five flawed copies, eleven weakened solutions that must each lose points in the matching check, and the fixtures.

python -m pytest tests/test_advlab.py | sed 's/ in [0-9.]*s//'

...............................                                          [100%]
31 passed

Pass checklist:

• Easy: you predicted 20 and FAIL with no gate failing, and can say why a perfect module is not a pass.
• Medium: only recommend_choice fails (2 points), the score is 98 and the result is PASS, and you can explain why that is not safe.
• Hard: question gives 100 points, no gate failure and FAIL through trace_sample.json; address gives one gate failure; you can argue for or against a sixth gate.
• Lab: 75 or more points, five green gates, all eleven submission items, and your own tests pass offline.

Summary

• The lab turns the Advanced Parts into six fixture-driven challenges worth 100 points; the pass mark is 75, with five critical gates and a complete submission.
• The gates (cross-tenant exposure, unauthorised tool run, credentials, reproducibility, rollback or shutdown) outrank the score, and each has a flawed copy that scores 100 and still fails.
• A gate must probe the way a leak happens: a shared object, recording tools, two fresh processes. A test that cannot fail proves nothing.
• The grader checks fixtures only, runs your code without a sandbox, and uses a toy embedder, synthetic labels and a scripted agent. Write your own cases and an honest limitations file.
• Real-model and container extensions are optional and never graded.

Next: Capstone, Cheat Sheet and What Next turns this lab into a finished portfolio project, collects the commands and the architecture on one page, and shows where to go after this tutorial.