Chapter 3 · Documentation and Oversight
Disclosures and Documentation Validation: Checking a System Card with Python
- Page 10 of 22
- 19 min read
The previous page built a card by hand. This page finishes the job: what to say about data permission and versions, how to write about what an output can and cannot tell you, how to tell the people affected in plain words, what to publish and what to keep internal, and how a short program can check a card. The checks are deliberately modest. They cannot tell whether a card is true; they can show where it is empty, unsupported or in conflict with itself.
You need Python 3.12 and its standard library only: no packages, no network, no model calls. This page runs on its own in a fresh folder, so it re-creates a shorter version of the card from Model Cards, System Cards and Datasheets: What to Document and for Whom.
Honesty convention. Every name, file, date and number here is invented, and results illustrate how a check works. A card that passes every check below is not thereby accurate, safe or lawful. The scripts report what is missing or unknown and never fill a gap with a guess. Nothing here is legal advice.
What you will learn
- How to record data sources, permission assumptions and the exact versions of the model and system.
- How to describe an output's limits without overclaiming that the system "explains" itself.
- How to write a user-facing AI-use disclosure in plain language, and how to tell a weak one from a better one.
- How to separate public documentation from internal records without hiding limitations.
- How to draft a card from a template, validate required fields, flag missing evidence and catch contradictions and stale claims.
Data sources, permissions and versions
Record data sources and permission assumptions. For each dataset, write where it came from and on what basis it may be used for this purpose. An assumption is something you believe but have not checked. Write it as an assumption, say who will check it, and leave "checked by" empty until someone has. Data Governance: Purpose, Consent, Minimisation, Quality and Re-Identification explains the underlying questions.
Record model and system versions. The model version identifies the trained model; the system version identifies the whole workflow around it, including the parser, thresholds and interface. Results belong to a version. A card that says "tested" without a version cannot be matched to what is running today.
Toy data: a short card for version 0.4 of the screening feature.
CARD = {
"name": "ShortlistHelper (hypothetical)",
"card_version": "1.2",
"versions": {"system": "0.4", "model": "fit-scorer-2026.08", "parser": "1.3"},
"intended_use": "order the CVs for ONE job so a recruiter knows which to read first",
"out_of_scope": ["automatic rejection", "ranking current staff"],
"data_sources": [
{"name": "past_applicants_2023_2025", "origin": "the agency's own archive",
"permission": {"basis": "assumed",
"assumption": "the 2023 privacy notice covers testing of tools",
"checked_by": None}},
],
}
from pprint import pprint
for key in ("versions", "data_sources"):
print(key, "->")
pprint(CARD[key], width=88, sort_dicts=False)versions ->
{'system': '0.4', 'model': 'fit-scorer-2026.08', 'parser': '1.3'}
data_sources ->
[{'name': 'past_applicants_2023_2025',
'origin': "the agency's own archive",
'permission': {'basis': 'assumed',
'assumption': 'the 2023 privacy notice covers testing of tools',
'checked_by': None}}]The permission is recorded as assumed with nobody having checked it. That is honest, and it is also a finding: a later check should flag it rather than let it pass as settled.
Output limits without overclaiming
Describe output limitations without overclaiming explainability. An explanation in this course means a statement that a person can use to understand and challenge a result. Many tools show something that looks like one: highlighted words, a ranked list of "factors", a short generated sentence. These show what the model reacted to, or text a model produced. Whether they reflect the real reason for a score is a separate question, and it needs its own test.
| Overclaim | Better wording |
|---|---|
| "The highlighted words are the reasons for the score." | "Highlighted words are those that changed the score most. We have not tested whether they match a human reason." |
| "The system explains every decision." | "The system shows a score and highlights. It gives no complete explanation of why a CV scored as it did." |
| "The score measures candidate quality." | "The score reflects similarity to the job text. It does not measure ability or likely success." |
CARD["output_limitations"] = {
"score_is": "a number from 0 to 100 computed by a model from the CV text",
"score_is_not": ["a measure of ability", "a probability of success", "a reason"],
"highlights": "words that changed the score most; not tested as a human reason",
}
print(CARD["output_limitations"]["score_is_not"])['a measure of ability', 'a probability of success', 'a reason']Telling the people affected
Write a user-facing AI-use disclosure. A disclosure tells the people affected, in words they can use, that an AI is involved. It should say what the system does, which data it uses, who makes the decision, what its limits are, and how to ask for a person. Rules about notices differ between countries and sectors (see Standards, Regulation and Requirements Mapping: What Applies, to Whom, and Why); a good notice also serves people where no rule applies.
Invented scenario: two versions of the notice on the agency's application form.
WEAK
Applications may be processed using automated tools in accordance with applicable policies
and procedures, and the outputs of such tools may inform decisions.
BETTER
We use a computer program to suggest the order in which our recruiters read CVs. It reads
only the text of your CV and gives it a score. A recruiter, not the program, decides who is
invited to an interview. A CV low on the list may not be opened unless a recruiter chooses
to. If you want a person to read your whole CV, tell us on the form or by email. The program
has been tested only on CVs written in English. We last checked this notice on 2026-10-01.The weak version is true and says almost nothing: no reader learns what the tool does, who decides or how to ask for a person. The better version is honest about the unread CVs, which also keeps it consistent with the card. A crude proxy for readability is the average sentence length; it is a signal only, because a short sentence can still mislead.
import re
def sentence_stats(text):
sentences = [s for s in re.split(r"(?<=[.!?])\s+", " ".join(text.split())) if s]
words = sum(len(s.split()) for s in sentences)
return len(sentences), words, round(words / len(sentences), 1)
weak = ("Applications may be processed using automated tools in accordance with applicable "
"policies and procedures, and the outputs of such tools may inform decisions.")
better = ("We use a computer program to suggest the order in which our recruiters read CVs. "
"It reads only the text of your CV and gives it a score. A recruiter, not the program, "
"decides who is invited to an interview. A CV low on the list may not be opened unless "
"a recruiter chooses to. If you want a person to read your whole CV, tell us on the "
"form or by email. The program has been tested only on CVs written in English. "
"We last checked this notice on 2026-10-01.")
for name, text in (("weak", weak), ("better", better)):
print(name, "(sentences, words, words per sentence):", sentence_stats(text))weak (sentences, words, words per sentence): (1, 23, 23.0)
better (sentences, words, words per sentence): (7, 91, 13.0)A notice should also exist in the languages your applicants use. The card says only English was tested; the notice must not suggest more.
Public documentation and internal records
Distinguish public documentation from internal restricted records. Some of what you know is for everyone who may be affected: what the system is for, its limits, how to reach a person. Some is restricted for good reasons: personal data, security details, commercial terms, the exact content of test sets that could be gamed. Two rules help. First, keep the reasons for restriction in writing. Second, never restrict a limitation because it is embarrassing. Restricted records are also still available to the people who must review them, such as auditors and, where the law says so, authorities.
CARD.update({
"tests": [{"name": "group comparison", "result": "done", "evidence": "eval/group_check.csv"},
{"name": "parser check", "result": "passed", "evidence": None}],
"evaluation": {"claim": "score order tested on 40 past applicants",
"evidence_file": "eval/shortlist_check_01.csv"},
"limitations": ["uses CV text only", {"status": "UNKNOWN", "why": "no calibration test run"}],
"language_coverage": {"tested": ["English"], "not_tested": ["Spanish"]},
"accessibility": {"status": "UNKNOWN", "why": "upload form not reviewed"},
"human_oversight": {"escalation": ["parse warning -> read by hand"]},
"failure_modes": [],
"next_review": "2026-09-30",
})
VISIBILITY = {"name": "public", "card_version": "public", "versions": "public",
"intended_use": "public", "out_of_scope": "public", "output_limitations": "public",
"limitations": "public", "language_coverage": "public", "accessibility": "public",
"human_oversight": "public", "next_review": "public",
"data_sources": "internal", "tests": "internal", "evaluation": "internal",
"failure_modes": "public"}
def view(card, audience):
keep = {k: v for k, v in card.items()
if audience == "internal" or VISIBILITY[k] == "public"}
return keep, len(card) - len(keep)
public, withheld = view(CARD, "public")
import textwrap
print(textwrap.fill("public sections: " + ", ".join(public), 90))
print("withheld from the public page:", withheld, "(listed as available to reviewers)")
assert "limitations" in public and "language_coverage" in publicpublic sections: name, card_version, versions, intended_use, out_of_scope,
output_limitations, limitations, language_coverage, accessibility, human_oversight,
failure_modes, next_review
withheld from the public page: 3 (listed as available to reviewers)The public view still holds every limitation, and it says that something is withheld. Putting test sets and data records in the internal view is a choice of this toy example; each team should write down why, and who may read the internal view. Failure modes are public here, because a known way to fail is a limitation that affected people need to know.
Draft and validate a card
Generate a draft system card from a structured template. A template lists the fields a card must have. A program can turn a dictionary into a readable draft, and it can show which fields are not yet filled. The draft writes what the dictionary says and never adds anything. Validate required fields in the same file: a field is MISSING if it is absent or empty, and UNKNOWN if someone has recorded a gap on purpose.
# file: cardcheck.py
import textwrap
from datetime import date
AS_OF = date(2026, 10, 12) # a fixed "today", so every run gives the same answer
MISSING = object()
REQUIRED = [
("name", "Feature"), ("versions.system", "System version"),
("versions.model", "Model version"), ("intended_use", "Intended use"),
("out_of_scope", "Out of scope"), ("data_sources", "Data sources"),
("evaluation.evidence_file", "Evaluation evidence"), ("limitations", "Limitations"),
("language_coverage", "Language coverage"), ("accessibility", "Accessibility"),
("human_oversight.escalation", "Escalation"), ("failure_modes", "Failure modes"),
("next_review", "Next review"),
]
def get(card, path):
node = card
for key in path.split("."):
if not isinstance(node, dict) or key not in node:
return MISSING
node = node[key]
return node
def status(value):
if value is MISSING or value in ("", [], {}, None):
return "MISSING"
if isinstance(value, dict) and value.get("status") == "UNKNOWN":
return "UNKNOWN"
return "ok"
def validate(card):
return [(status(get(card, p)), p, label) for p, label in REQUIRED]
def unknown_paths(x, prefix=""):
found = []
if isinstance(x, dict):
if x.get("status") == "UNKNOWN":
found.append(prefix)
for k, v in x.items():
found += unknown_paths(v, f"{prefix}.{k}" if prefix else k)
elif isinstance(x, list):
for i, v in enumerate(x):
found += unknown_paths(v, f"{prefix}[{i}]")
return found
def show(value):
if isinstance(value, str):
return value
if isinstance(value, list):
return "; ".join(show(v) for v in value)
if isinstance(value, dict) and value.get("status") == "UNKNOWN":
return "UNKNOWN (" + value["why"] + ")"
if isinstance(value, dict):
return ", ".join(f"{k}: {show(v)}" for k, v in value.items())
return str(value)
def draft(card):
lines = ["SYSTEM CARD (DRAFT, not reviewed)"]
for path, label in REQUIRED:
value, s = get(card, path), status(get(card, path))
if s == "MISSING":
text = "MISSING - the owner must supply this"
elif s == "UNKNOWN":
text = "UNKNOWN - " + value["why"]
else:
text = show(value)
lines.append(textwrap.fill(f"{label}: {text}", 92, subsequent_indent=" "))
return "\n".join(lines)from cardcheck import draft, validate, unknown_paths
print(draft(CARD))
rows = validate(CARD)
print()
print({s: sum(1 for r in rows if r[0] == s) for s in ("ok", "UNKNOWN", "MISSING")})
for s, path, label in rows:
if s != "ok":
print(f"{s:<8}{path}")
print("UNKNOWN anywhere in the card:", sorted(unknown_paths(CARD)))SYSTEM CARD (DRAFT, not reviewed)
Feature: ShortlistHelper (hypothetical)
System version: 0.4
Model version: fit-scorer-2026.08
Intended use: order the CVs for ONE job so a recruiter knows which to read first
Out of scope: automatic rejection; ranking current staff
Data sources: name: past_applicants_2023_2025, origin: the agency's own archive, permission:
basis: assumed, assumption: the 2023 privacy notice covers testing of tools, checked_by:
None
Evaluation evidence: eval/shortlist_check_01.csv
Limitations: uses CV text only; UNKNOWN (no calibration test run)
Language coverage: tested: English, not_tested: Spanish
Accessibility: UNKNOWN - upload form not reviewed
Escalation: parse warning -> read by hand
Failure modes: MISSING - the owner must supply this
Next review: 2026-09-30
{'ok': 11, 'UNKNOWN': 1, 'MISSING': 1}
UNKNOWN accessibility
MISSING failure_modes
UNKNOWN anywhere in the card: ['accessibility', 'limitations[1]']The draft is readable but not finished, and it says so on every line that matters. Failure modes is MISSING (the list is empty), Accessibility is UNKNOWN on purpose, and one limitation deeper in the card is UNKNOWN too. A validator that filled these with plausible sentences would produce a card that looks complete and means nothing.
Flag missing evidence and check for contradictions
Flag missing evidence instead of filling it with guesses. A card makes claims such as "tested" or "passed". Each should point to something a reader can open. The check below flags a claim with no evidence reference, a reference to a file that does not exist, and a permission that is still an unchecked assumption.
Review documentation for contradictions and outdated claims. Documentation goes stale in two ways: the system changes, or time passes. Compare the card with other records that should agree with it: a release log, a configuration, and the review date. The checks are simple rules; a person decides what each flag means.
# file: consistency.py
from datetime import date
from pathlib import Path
from cardcheck import AS_OF, get
def evidence_flags(card):
flags = []
path = get(card, "evaluation.evidence_file")
if isinstance(path, str) and not Path(path).exists():
flags.append(f"evaluation: evidence file {path} not found")
for t in card.get("tests", []):
ev = t.get("evidence")
if not ev:
flags.append(f"test '{t['name']}' says '{t['result']}' but cites no evidence")
elif not Path(ev).exists():
flags.append(f"test '{t['name']}': evidence file {ev} not found")
for s in card.get("data_sources", []):
p = s.get("permission", {})
if p.get("basis") == "assumed" and not p.get("checked_by"):
flags.append(f"data '{s['name']}': permission is an unchecked assumption")
return flags
def consistency_flags(card, release, config):
flags = []
for part in ("system", "model"):
said, real = card["versions"][part], release[part]
if said != real:
flags.append(f"{part} version: card says {said}, release log says {real}")
due = date.fromisoformat(card["next_review"])
if due < AS_OF:
flags.append(f"review overdue: {due} is {(AS_OF - due).days} days before {AS_OF}")
if config.get("auto_reject_below") is not None and \
"automatic rejection" in card.get("out_of_scope", []):
flags.append("card puts automatic rejection out of scope, but the config sets "
f"auto_reject_below={config['auto_reject_below']}")
tested = set(card["language_coverage"]["tested"])
for lang in sorted(set(config["languages_enabled"]) - tested):
flags.append(f"language {lang} is enabled in the config but not tested")
return flagsfrom pathlib import Path
from consistency import evidence_flags, consistency_flags
Path("eval").mkdir()
Path("eval/shortlist_check_01.csv").write_text("toy stand-in for the team's results file\n")
RELEASE = {"system": "0.5", "model": "fit-scorer-2026.08"} # Toy data: the release log
CONFIG = {"auto_reject_below": 15, "languages_enabled": ["English", "Spanish"]} # Toy data
print("EVIDENCE")
for f in evidence_flags(CARD):
print(" -", f)
print("CONSISTENCY (as of 2026-10-12)")
for f in consistency_flags(CARD, RELEASE, CONFIG):
print(" -", f)EVIDENCE
- test 'group comparison': evidence file eval/group_check.csv not found
- test 'parser check' says 'passed' but cites no evidence
- data 'past_applicants_2023_2025': permission is an unchecked assumption
CONSISTENCY (as of 2026-10-12)
- system version: card says 0.4, release log says 0.5
- review overdue: 2026-09-30 is 12 days before 2026-10-12
- card puts automatic rejection out of scope, but the config sets auto_reject_below=15
- language Spanish is enabled in the config but not testedThe evidence flags are about claims the card cannot back up today. The consistency flags show four kinds of conflict. The most serious is the contradiction between the card (automatic rejection is out of scope) and the configuration (a threshold below which applicants are rejected, apparently without a person). Neither flag says which document is right. Finding out is the owner's job, and the answer may be to change the configuration, the card, or the plan.
Limit of these checks. They compare what is written. A false statement that is written consistently in every place passes. Clean output means "no conflict found by these rules", not "the documentation is true". Real review still needs people who can look at the system.
Where this shows up at work
Teams run checks like these before a release or a quarterly review and attach the output to the record. Auditors and customers often ask for the card, the release log and the review history together. A flagged gap with an owner and a date is normal; an unflagged gap discovered by someone else is the expensive kind.
Common mistakes
- Writing "compliant" or "bias-free". Neither word is something a card can prove. Describe what was done and where the evidence is.
- Hiding limits as "internal". People affected need the limits that change what they do. Restrict personal data and security details, not weaknesses.
- Treating assumptions as facts. "Covered by our privacy notice" is an assumption until someone with the right knowledge has checked it.
- A validator that fills blanks. If a program invents text to pass, it destroys the signal. Report MISSING and UNKNOWN as they are.
- Trusting a clean check. Passing the rules proves only that nothing conflicted. Someone still has to check the system against the card.
Try it yourself
Use the namespace from this page. The exercises revisit Policies, Vendors, Records and Incidents: Responsible AI in a Small Team from the Beginner tutorial.
- Easy. Which of these overclaim explainability? (a) "The highlighted words are the reasons for the score." (b) "Highlighted words changed the score most; we have not tested them as reasons." (c) "The model explains every decision it makes." Say why.
- Medium. Rewrite the weak notice "AI may be used in our process." so that it says what the tool does, who decides, one limit and how to ask for a person.
- Hard. Produce a system card for the toy feature below, in a dictionary that
validateaccepts, then run the validator and the evidence check.
Invented scenario:
Feature: ReplyDrafter (hypothetical), version 0.1, model "drafter-2026.09".
Owner: Kofi, who runs a small shop in Kumasi. Efua, a staff member, reads every draft
before it is sent. If a customer complains, Kofi handles it.
Use: drafts English replies to customer emails about orders. It must not answer
refund or legal questions; those go to Kofi.
Tested: on 25 past emails (Toy data) by Kofi himself; results in eval/reply_check.csv.
3 of the 25 drafts stated the refund policy wrongly.
Languages: English tested. Twi (a language widely spoken in Ghana) not tested.
Data: past customer emails from the shop's inbox; customers have not been asked.
Access: nobody has checked whether customers can reach the shop by another route.
Review: next card review 2026-12-01.Answers
Easy. (a) and (c) overclaim. (a) treats highlights as reasons without a test; (c) claims complete explanation. (b) says what the highlights are and what is not known.
Pass checklist: names both overclaims and the reason (no test of the link between highlights and reasons; "every" is absolute). A weak answer rejects all three because they contain the word "explain".
Medium. A strong answer: "We use a computer program to draft the first version of some replies to our customers. It reads only your message. A staff member reads and changes every draft before it is sent. It can make mistakes, so please tell us if something looks wrong. To ask for a person, write to the address below." It names what the tool does, who decides, a limit and the route to a person.
Pass checklist: plain words and short sentences; the human who decides is named as a role; one honest limit; a way to reach a person. A weak answer repeats "automated tools" or promises accuracy.
Hard.
from pathlib import Path
from cardcheck import validate
REPLY = {
"name": "ReplyDrafter (hypothetical)",
"versions": {"system": "0.1", "model": "drafter-2026.09"},
"intended_use": "draft English replies to order emails; a staff member reads every draft",
"out_of_scope": ["answering refund or legal questions", "sending without a person reading"],
"data_sources": [{"name": "past_customer_emails", "origin": "the shop's inbox",
"permission": {"basis": "assumed",
"assumption": "customers were not asked; not yet checked",
"checked_by": None}}],
"evaluation": {"claim": "25 past emails tested by the owner",
"evidence_file": "eval/reply_check.csv"},
"tests": [{"name": "past-email test", "result": "done", "evidence": "eval/reply_check.csv"}],
"limitations": ["3 of 25 drafts stated the refund policy wrongly (Toy data)"],
"language_coverage": {"tested": ["English"], "not_tested": ["Twi"]},
"accessibility": {"status": "UNKNOWN", "why": "no check of other contact routes"},
"human_oversight": {"escalation": ["refund or legal question -> Kofi"]},
"failure_modes": [{"mode": "wrong refund policy", "signal": "Efua spots it or a complaint",
"response": "Kofi handles it"}],
"next_review": "2026-12-01",
}
Path("eval/reply_check.csv").write_text("toy stand-in for Kofi's results file\n")
print([r[:2] for r in validate(REPLY) if r[0] != "ok"])
from consistency import evidence_flags
print(evidence_flags(REPLY))[('UNKNOWN', 'accessibility')]
["data 'past_customer_emails': permission is an unchecked assumption"]Pass checklist: every required field is filled from the scenario or marked UNKNOWN; the wrong refund statements appear as a limitation and a failure mode; the permission is recorded as an unchecked assumption; the scenario's missing accessibility check is UNKNOWN, not "ok". Here the validator finds one deliberate gap, and the evidence check names the permission assumption. A weak answer writes "customers consented" or "accessible", which the scenario does not support.
Summary
- Record data sources with permission as an assumption until someone checks it, and tie every result to a model and system version.
- Describe highlights and scores for what they are; do not claim an explanation you have not tested.
- A good disclosure says what the system does, what data it uses, who decides, its limits and how to reach a person, in short plain sentences.
- Keep personal data and security details internal; never hide a limitation that affected people need.
- Validate required fields, report MISSING and UNKNOWN, flag claims without evidence and compare the card with the release log, configuration and review date.
Next: Human Oversight Design: Meaningful Review, Not Rubber-Stamping asks whether the people named in a card can really review, override and appeal an AI-assisted decision.