Chapter 1 · Risk and Frameworks
Principles and Frameworks in Practice: From Values to Testable Controls
- Page 3 of 22
- 17 min read
Principles such as fairness, transparency and accountability are easy to agree with and hard to check. A team that says "we follow responsible-AI frameworks" has said little until each concern is tied to something a person does and a record that shows it. This page practises that step. The ideas themselves are in the Beginner pages Core Ethical Principles: Fairness, Transparency, Privacy, Safety and Accountability and Frameworks, Standards and Competing Principles: Ethics, Law and Policy; here you apply them, and keep the mapping as data that a program can check.
You need Python 3.12 and its standard library only. Each page runs in a fresh folder, so files are created with # file: blocks. Type the blocks in order into one session; the text under a block is its real output.
Honesty convention. Every dataset is synthetic and labelled
Toy data:; every company, person and number is invented. Results show how a calculation works, not what is true of any real system. The frameworks named here are real, and their dates come from the owners' pages as checked on 2026-10-12; they change, so check the owner's page before you cite one. Nothing here is legal advice.
What you will learn
- How to turn an ethical concern into a testable control objective with evidence.
- How to read a product scenario through the OECD AI Principles, UNESCO's Recommendation and the four NIST AI RMF functions, and how those readings differ.
- How to use the NIST Generative AI Profile as a list of risk areas, not as a checklist to tick.
- How to tell guidance from a certification requirement, and what ISO/IEC 42001's management-system idea adds.
- How to keep a concern-to-control mapping as data and let a script flag what is unmapped or unevidenced.
The scenario
Invented scenario:
Company: Harbourline Staffing (invented), the agency from pages 1 and 2.
Change: ShortlistHelper v2 keeps the ranked list and adds a generated 3-line summary of each CV.
Claim: The agency owner says: "We follow responsible-AI frameworks."
Question: What does that mean, concern by concern, and what could we show an outside reader?Three things are easy to confuse. An ethical concern is something a thoughtful person worries about. A framework principle is a shared, voluntary way of naming it. A control is what your organisation does about it. A claim is only as strong as the last link you can show.
From concern to control objective
Map an ethical concern to a control objective. A control objective says what a control must achieve in a form you can test. "Be fair" is a value; it cannot fail a test. A usable objective answers five questions.
A control objective answers five questions:
Who checks what, how often, what counts as a pass, and which record proves it.| Concern | Weak objective | Testable objective |
|---|---|---|
| Scores penalise career breaks | "The tool should be fair" | Before each release, selection rates by group are computed on a held-out set; a named person signs any gap above the team's chosen limit |
| CV text kept beyond hiring | "We protect privacy" | CV text is deleted within the retention period written in the contract; the Operations manager checks a deletion log each quarter |
| Recruiters trust the ranking | "Avoid over-trust" | Each week a recruiter reads 50 random low-ranked CVs and records how many were wrongly low |
The strong versions name a person, a frequency, a pass condition and a record. The limit in the first row is a choice the team must justify; a framework does not give it to you.
Keep the mapping as data
Create three files. frameworks.py holds the names exactly as the owners list them (so a typo is caught), controls.json holds the mapping, and mapcheck.py checks it and reports gaps. The codes O1 to O5 and U1 to U10 are ours. The OECD AI Principles were adopted in 2019 and updated in May 2024; UNESCO's Recommendation on the Ethics of AI was adopted in November 2021; NIST AI RMF 1.0 was released on 26 January 2023, and NIST AI 600-1 (the Generative AI Profile) in July 2024.
# file: frameworks.py
# Names as listed on the owners' pages, checked 2026-10-12. The short codes (O1, U1 ...) are ours.
OECD = { # OECD AI Principles: adopted 2019, updated May 2024
"O1": "inclusive growth, sustainable development and well-being",
"O2": "human rights and democratic values, including fairness and privacy",
"O3": "transparency and explainability",
"O4": "robustness, security and safety",
"O5": "accountability",
}
UNESCO = { # Recommendation on the Ethics of AI, November 2021: the ten principles
"U1": "proportionality and do no harm",
"U2": "safety and security",
"U3": "privacy and data protection",
"U4": "multi-stakeholder governance",
"U5": "responsibility and accountability",
"U6": "transparency and explainability",
"U7": "human oversight",
"U8": "sustainability",
"U9": "awareness and literacy",
"U10": "fairness and non-discrimination",
}
NIST_FUNCTIONS = ("Govern", "Map", "Measure", "Manage") # NIST AI RMF 1.0, 26 January 2023
GENAI_RISKS = ( # NIST AI 600-1, July 2024: the twelve risk areas it names
"CBRN information or capabilities", "confabulation", "dangerous, violent or hateful content",
"data privacy", "environmental impacts", "harmful bias or homogenisation", "human-AI configuration",
"information integrity", "information security", "intellectual property",
"obscene, degrading and/or abusive content", "value chain and component integration",
)
FRAMEWORKS = { # name: (kind, version or date, binding on its own?)
"OECD AI Principles": ("intergovernmental recommendation", "adopted 2019, updated May 2024", False),
"UNESCO Recommendation": ("UN agency recommendation", "November 2021", False),
"NIST AI RMF": ("US government framework", "1.0, 26 January 2023", False),
"NIST AI 600-1": ("profile of the NIST AI RMF", "July 2024", False),
"ISO/IEC 42001": ("paid management-system standard", "check iso.org for the edition", False),
}Toy data: the mapping for ShortlistHelper v2. The concerns echo the page 1 register; K5 is new, because of the generated summaries. Evidence is left empty where none exists. In the JSON file, leave out the first line, // file: controls.json; it only names the file.
// file: controls.json
{
"scenario": "ShortlistHelper v2 (invented): ranks CVs and writes a three-line summary of each CV",
"concerns": [
{"id": "K1", "concern": "Scores penalise career breaks", "oecd": "O2", "unesco": "U10",
"nist": "Measure", "genai": "harmful bias or homogenisation",
"control": "group check per release", "evidence": ""},
{"id": "K2", "concern": "Recruiters trust the ranking", "oecd": "O5", "unesco": "U7",
"nist": "Manage", "genai": "human-AI configuration",
"control": "weekly sample of low ranks", "evidence": "internal test"},
{"id": "K3", "concern": "CV text kept beyond hiring", "oecd": "O2", "unesco": "U3",
"nist": "Map", "genai": "data privacy",
"control": "contract retention limit", "evidence": ""},
{"id": "K4", "concern": "Applicants cannot ask why", "oecd": "O3", "unesco": "U6",
"nist": "Govern", "genai": "",
"control": "notice and review route", "evidence": "draft notice"},
{"id": "K5", "concern": "Summaries invent CV details", "oecd": "O4", "unesco": "U2",
"nist": "Measure", "genai": "confabulation",
"control": "summary test on sample CVs", "evidence": ""},
{"id": "K6", "concern": "No route for offline applicants", "oecd": "O2", "unesco": "U10",
"nist": "Map", "genai": "",
"control": "", "evidence": ""},
{"id": "K7", "concern": "Staff worry about job changes", "oecd": "", "unesco": "",
"nist": "", "genai": "",
"control": "", "evidence": ""}
]
}# file: mapcheck.py
import json
import textwrap
from frameworks import OECD, UNESCO, NIST_FUNCTIONS, GENAI_RISKS
def load(path):
with open(path, encoding="utf-8") as f:
return json.load(f)["concerns"]
def flags(c):
"""Problems with one row. Nothing is guessed or filled in."""
if not (c["oecd"] or c["unesco"] or c["nist"]):
return ["unmapped: no principle or function chosen yet"]
out = []
for key, allowed in (("oecd", OECD), ("unesco", UNESCO), ("nist", NIST_FUNCTIONS), ("genai", GENAI_RISKS)):
if c[key] and c[key] not in allowed:
out.append(f"{key} '{c[key]}' is not in that framework's list")
if not c["control"]:
out.append("no control objective")
elif not c["evidence"]:
out.append("MISSING evidence")
return out
def report(rows):
print(f"{'id':<4}{'concern':<33}{'OECD':<5}{'UNESCO':<7}{'NIST':<8}{'control':<27}evidence")
for c in rows:
print(f"{c['id']:<4}{c['concern']:<33}{c['oecd'] or '-':<5}{c['unesco'] or '-':<7}{c['nist'] or '-':<8}"
f"{c['control'] or '-':<27}{c['evidence'] or 'MISSING'}")
print()
for c in rows:
for text in flags(c):
print(f"{c['id']}: {text}")
print()
print("NIST functions:", ", ".join(f"{n} {sum(c['nist'] == n for c in rows)}" for n in NIST_FUNCTIONS))
named = {c["genai"] for c in rows if c["genai"]}
print(f"GenAI Profile risk areas named: {len(named)} of {len(GENAI_RISKS)}")
rest = "; ".join(r for r in GENAI_RISKS if r not in named)
print(textwrap.fill("not yet considered: " + rest, width=96, subsequent_indent=" "))import mapcheck
rows = mapcheck.load("controls.json")
mapcheck.report(rows)id concern OECD UNESCO NIST control evidence
K1 Scores penalise career breaks O2 U10 Measure group check per release MISSING
K2 Recruiters trust the ranking O5 U7 Manage weekly sample of low ranks internal test
K3 CV text kept beyond hiring O2 U3 Map contract retention limit MISSING
K4 Applicants cannot ask why O3 U6 Govern notice and review route draft notice
K5 Summaries invent CV details O4 U2 Measure summary test on sample CVs MISSING
K6 No route for offline applicants O2 U10 Map - MISSING
K7 Staff worry about job changes - - - - MISSING
K1: MISSING evidence
K3: MISSING evidence
K5: MISSING evidence
K6: no control objective
K7: unmapped: no principle or function chosen yet
NIST functions: Govern 1, Map 2, Measure 2, Manage 1
GenAI Profile risk areas named: 4 of 12
not yet considered: CBRN information or capabilities; dangerous, violent or hateful content;
environmental impacts; information integrity; information security; intellectual property;
obscene, degrading and/or abusive content; value chain and component integrationThe script reports; it does not repair. K1, K3 and K5 have a control objective but no evidence: nobody has yet shown that the check runs. K6 has a principle but no control. K7 has been mapped to nothing, and that is information too: some concerns do not fit a tidy list, and the right response is to discuss them, not to force a label. "MISSING" is a finding, not a failure of the team; inventing the evidence would be.
OECD and UNESCO on the scenario
Apply the OECD AI Principles to a product scenario. The OECD AI Principles have five values-based principles and five recommendations to governments, which a product team seldom applies directly. They are non-binding. The scenario reads like this.
| OECD principle | A question for ShortlistHelper v2 | Concern |
|---|---|---|
| O1 inclusive growth, sustainable development and well-being | Does it widen or narrow who gets interviews? | K6 (K7) |
| O2 human rights and democratic values, including fairness and privacy | Are applicants treated fairly, and is their data protected? | K1, K3, K6 |
| O3 transparency and explainability | Can an applicant learn that a tool was used and ask why? | K4 |
| O4 robustness, security and safety | Do summaries stay accurate and the data stay safe? | K5 |
| O5 accountability | Is a named role answerable for the ranking? | K2 |
Map a scenario to UNESCO's Ethics Recommendation. UNESCO's text, adopted by its 193 Member States, rests on four values and ten principles; it is also non-binding. The ten are listed in frameworks.py. It cuts the same ground differently: human oversight (U7) and privacy and data protection (U3) are principles of their own, while the OECD list names privacy inside its human-rights principle (O2). Mapping K2 and K3 shows it: the same concerns land on O5 and O2 in one framework and on U7 and U3 in the other. Neither mapping is wrong. Choose the vocabulary your audience (customers, auditors, regulators, colleagues abroad) already uses, and say which one you chose.
Principles also pull against each other. To check whether scores differ by group, a team must hold group data, but privacy points toward holding less. Teams, and countries, settle this differently; record what you chose and who decided. Law may constrain the choice where you work, so ask a specialist rather than assume.
The four NIST AI RMF functions
NIST AI RMF 1.0 is a US government framework, intended for voluntary use and usable anywhere. As of 2026-10-12 NIST says it is being revised, with no timeline and no new version published; a 2026 concept note concerns a profile for critical infrastructure and is not a new framework. Record the version you used. The one-line meanings below are our wording, not NIST's.
| Function (our gloss) | A question for the scenario | What you would keep |
|---|---|---|
| Govern (policies, roles, accountability) | Who may approve a release, and who answers for it? Do applicants get a notice and a review route (K4)? | Policy, role list, approved notice |
| Map (context and risks) | Who is affected, including applicants who cannot apply online (K6)? Where does CV text go and stay (K3)? | Scope, stakeholder list, data-flow note |
| Measure (test and track) | What is the group gap in who is shortlisted (K1)? How often is a summary wrong (K5)? | Test sets, results, thresholds with reasons |
| Manage (respond and prioritise) | What happens when the sample finds a wrongly low rank (K2)? Who can pause the tool? | Stop rule, incident log, review dates |
The last lines of the report count the mapped concerns per function. Here every function has at least one. A function with none is a prompt for a question, not an automatic failure: ask whether the work happens elsewhere or is not done.
The Generative AI Profile as a risk reference
Examine the NIST Generative AI Profile as a risk reference. NIST AI 600-1 is a cross-sector profile of the AI RMF, also voluntary. It names twelve risk areas, listed in GENAI_RISKS. Used as a vocabulary, it helps a team notice what it has not considered: the report shows four areas named and eight not yet considered, such as information security, intellectual property and the supply chain of components. Not every area applies to every feature. The honest step for each unnamed area is to decide relevant or not relevant, with a reason; this page does not reproduce what the document says about each area, so read it before you use it.
Guidance, certification and ISO/IEC 42001
Distinguish framework guidance from a certification requirement. The OECD, UNESCO and NIST texts are voluntary guidance: nothing in them is a legal duty by itself. What you may write depends on the evidence you hold.
| Wording | What it needs |
|---|---|
| used as a reference | Nothing more than honesty: say which version and how |
| aligned with | Your own written mapping, as in controls.json |
| certified to | A certificate from an outside body, for a standard that has such a scheme |
| compliant with | Binding text that applies to you (for example a law), and a legal check |
Toy data: five invented marketing claims, checked against the table.
from frameworks import FRAMEWORKS
def verdict(name, wording):
kind, version, binding = FRAMEWORKS[name]
if wording == "used as a reference":
return f"ok: name the version ({version})"
if wording == "aligned with":
return "ok only with a written mapping like controls.json"
if wording == "compliant with":
return "ok only for binding text" if binding else "no: not binding on its own, so not 'compliance'"
return "unverified: needs an outside certificate; check the owner"
CLAIMS = [("OECD AI Principles", "compliant with"), ("NIST AI RMF", "certified to"),
("NIST AI RMF", "aligned with"), ("ISO/IEC 42001", "certified to"),
("NIST AI 600-1", "used as a reference")]
for name, wording in CLAIMS:
print(f"{wording} {name}".ljust(40), verdict(name, wording))compliant with OECD AI Principles no: not binding on its own, so not 'compliance'
certified to NIST AI RMF unverified: needs an outside certificate; check the owner
aligned with NIST AI RMF ok only with a written mapping like controls.json
certified to ISO/IEC 42001 unverified: needs an outside certificate; check the owner
used as a reference NIST AI 600-1 ok: name the version (July 2024)The script cannot know whether a certificate exists; it says "unverified" and sends you to the owner. The next page treats law and standards in more detail.
Examine ISO/IEC 42001's management-system approach. ISO/IEC 42001 is described by standards bodies as an AI management-system standard. It is a paid standard that this page cannot reproduce, and ISO's own site could not be checked for this page, so check iso.org for the current edition and status. In general, a management-system standard asks about the organisation's processes, not about one product: roles, a risk process, records, reviews, and acting on findings. Organisations may be audited by accredited certification bodies; that is general practice for ISO standards, not something confirmed for this standard here. A certificate would speak about the organisation's process. It would not be a certificate that a model is safe.
| Question | Product-level control | Management-system view |
|---|---|---|
| Subject | This feature, this release | How the organisation runs AI work generally |
| Example | The group check before release (K1) | A policy that requires such checks, a role that owns them, records kept, a yearly review |
| Evidence | The check report | Policy, role list, records and review minutes |
Where this shows up at work
Governance, product and procurement staff meet this when a customer asks "which framework do you follow?" or a tender asks for evidence. A mapping like controls.json answers honestly: what is mapped, what is evidenced, and what is open.
Common mistakes
- Mapping to the principle and stopping. "We follow O2" says nothing until a control and a record exist. Complete the chain.
- Writing "compliant" for voluntary guidance. It claims a legal status that does not exist. Write "aligned with" and show the mapping.
- Treating the twelve GenAI risk areas as a checklist. They are a vocabulary; record relevance and reasons.
- Forcing an unmapped concern into a label. Leave it flagged and discuss it.
- Citing a version without a date. Frameworks are revised; record the version and the review date.
Try it yourself
Use the namespace from this page. The exercises return to the page 1 register, where K3 is risk R4.
- Easy. A new row has
oecd"O7" andnist"Monitor". Predict whatmapcheck.flagsreports, then run it. - Medium. Map K7 ("Staff worry about job changes"): choose an OECD principle, a UNESCO principle, a NIST function and a testable control objective, and say what is still missing.
- Hard. Write code that lists the Generative AI Profile risk areas that have neither a concern nor a written reason in a
NOT_RELEVANTdictionary. Fill in reasons for the areas that clearly do not apply to a CV tool, and say which areas still need a decision.
Answers
Easy. Both labels are outside their lists.
bad = {"id": "K8", "concern": "Test", "oecd": "O7", "unesco": "", "nist": "Monitor", "genai": "", "control": "x", "evidence": "y"}
print(mapcheck.flags(bad))["oecd 'O7' is not in that framework's list", "nist 'Monitor' is not in that framework's list"]Pass checklist: both unknown labels found; understanding that the script checks names, not whether a mapping is sensible. A weak answer expects the script to judge quality.
Medium. One defensible mapping: O1 (well-being), U9 (awareness and literacy), Govern, with the objective "staff are briefed before the pilot and a named manager records their questions".
rows[6].update(oecd="O1", unesco="U9", nist="Govern", control="staff briefing before pilot")
print(mapcheck.flags(rows[6]))['MISSING evidence']The unmapped flag is gone; evidence is still missing, because a briefing has not happened. Pass checklist: valid labels; a testable objective with a person and a record; evidence left empty; a note that other mappings are reasonable. A weak answer invents evidence or says "mapped, so done".
Hard.
NOT_RELEVANT = { # our judgement, to revisit if the scope changes
"CBRN information or capabilities": "a CV tool has no such content",
"dangerous, violent or hateful content": "output is a three-line summary of a CV",
"obscene, degrading and/or abusive content": "output is a three-line summary of a CV",
}
named = {c["genai"] for c in rows if c["genai"]}
undecided = [r for r in mapcheck.GENAI_RISKS if r not in named and r not in NOT_RELEVANT]
print(len(undecided), "areas still need a decision:")
for r in undecided:
print(" -", r)5 areas still need a decision:
- environmental impacts
- information integrity
- information security
- intellectual property
- value chain and component integrationPass checklist: reasons are specific to the tool; areas that touch the tool (such as information integrity and information security) stay undecided; environmental impacts is not dismissed without assessment. A weak answer marks everything "not relevant" to empty the list.
Summary
- A usable control objective names who checks what, how often, the pass condition and the record.
- OECD, UNESCO and NIST give different vocabularies for the same concern; choose one, state it and record versions and dates.
- The four NIST functions are a way to ask questions, and the Generative AI Profile's twelve areas a list of risks to consider, both voluntary.
- Guidance, certification and legal compliance are different claims; write only the one you can evidence.
- ISO/IEC 42001 is about the organisation's management system; check iso.org for its status. A script should flag unmapped and unevidenced concerns, and never invent evidence.
Next: Standards, Regulation and Requirements Mapping: What Applies, to Whom, and Why separates standards from law and maps a feature to requirements and evidence.