Fairness and Its Trade-Offs: Inclusion, Accessibility and Language

On the previous page you learned to find a difference between groups. The next question is harder: is that difference unfair, and what should you do about it? Here the answer is rarely a single number. "Fair" can mean treating everyone by the same rule, giving everyone the same results, making the same kind of mistake equally often for every group, or making a good result mean the same thing for everyone. These goals often cannot all be met at once, so a team has to choose, say why, and watch the people who bear the cost of the choice.

This page keeps the ideas conceptual. You will use small invented counts to see how the trade-offs behave, but you do not need to calculate anything yourself. Inclusion, accessibility and language, which you met on Where Bias Comes From: Data, Design, Proxies and Feedback Loops, return here as questions about how a rule treats people who start from different places. The page ends with a hiring-screening scenario, a fairer way to run it, and an exercise on fairness trade-offs.

What you will learn

• The difference between treating people the same and giving them the same results, and why accessibility shows that both matter.
• What false positives and false negatives are in a real decision, and why their costs differ from one task to another.
• Four fairness goals in plain words, why they can conflict, and why demographic parity or equal error rates do not fit every task.
• How to question a proxy variable, and how to handle small groups and uncertain measurements.
• How to evaluate a hypothetical hiring-screening workflow and replace an exclusionary step with a fairer one.

Equal treatment and equal outcomes

Equal treatment means everyone is handled by the same rule. Equal outcomes means the results come out the same across groups. Each sounds fair, and each can be unfair when used alone.

Invented scenario:

A training company in Accra screens applicants with one online test. Everyone gets the same
30 minutes and the same page layout. Tunde, who is blind, uses a screen reader. The test shows
its questions as images with no text, so he cannot read them in time. Zainab has a fast
connection and a quiet room. Both were "treated equally". Their chances were not equal.

A common fix keeps the rule's purpose but changes how it is applied. Tunde gets a text version of the test and extra time if he needs it. The aim is the same (measure the skill), and the treatment differs so that the chance is equal. Reasonable people disagree about how far to go, and the next sections show why.

False positives and false negatives in a decision

Most screening decisions can be wrong in two ways. A false positive means the tool says "yes" and the truth is "no". A false negative means the tool says "no" and the truth is "yes". "Positive" is only a label for the tool's flag, so always say in words what the flag is.

The same tool cannot make both mistakes disappear. If you make it advance more people, you usually get fewer false negatives and more false positives. Which mistake is worse depends on the task, on who pays, and on whether the person can correct it. That is a value judgement, not a calculation, and it belongs in the decision pattern at "Benefits and harms". Do the two mistakes also fall equally on every group? That question leads to the next section.

Counts also matter because rare events mislead. Toy data:

A bank's alert tool checks 1,000 payments. In truth 20 are fraud and 980 are honest.
The tool flags 60 payments. 18 of them are real fraud and 42 are honest customers.
It misses 2 frauds.

Real fraud caught (18 of 20):        90%
Honest payments blocked (42 of 980): about 4.3%
Of the 60 flagged, truly fraud:      30%

The tool catches nine in ten frauds and still blocks 42 honest customers, because honest payments are so common. A flag can look accurate and still be wrong most of the time when the thing you are looking for is rare. (The numbers are invented and only show the arithmetic.)

Fairness goals that can conflict

Researchers have written down several precise definitions of group fairness. Here are four of them in plain words. In each, we compare groups of people, such as Group A and Group B, with one tool.

The first three definitions are from the research literature on fairness in machine learning (Hardt, Price and Srebro, 2016). The fourth is the standard way the literature describes the idea of predictive parity, and it is a plain-words gloss, not a quotation. These are measures you can compute on a dataset. They describe a tool on one dataset and do not prove that a real system is fair.

Now see what happens with one set of invented counts. Toy data:

A tool screens applicants for a warehouse-supervisor course.
Group A: 200 applicants, 120 truly suited (60%).  Group B: 100 applicants, 30 truly suited (30%).
"Truly suited" was decided later, by a trial month, so we know it for these invented people.

Rule 1: one cutoff for everyone (equal treatment)
  Group A: selected 112 of 200 (56%)   suited who were selected: 80%   unsuited selected: 20%
           of those selected, truly suited: 85.7%
  Group B: selected  28 of 100 (28%)   suited who were selected: 66.7%  unsuited selected: 11.4%
           of those selected, truly suited: 71.4%

Rule 2: cutoffs set so both kinds of error match (equalised odds)
  Group A: selected 112 of 200 (56%)   suited who were selected: 80%   unsuited selected: 20%
           of those selected, truly suited: 85.7%
  Group B: selected  38 of 100 (38%)   suited who were selected: 80%   unsuited selected: 20%
           of those selected, truly suited: 63.2%

Rule 3: cutoffs set so the same share is selected (demographic parity)
  Group A: selected 112 of 200 (56%)   suited who were selected: 80%   unsuited selected: 20%
           of those selected, truly suited: 85.7%
  Group B: selected  56 of 100 (56%)   suited who were selected: 86.7%  unsuited selected: 42.9%
           of those selected, truly suited: 46.4%

Rule 1 treats both groups the same but the suited people in Group B are selected less often (66.7% against 80%). Rule 2 fixes that, and now the error rates match, but 38% of Group B and 56% of Group A are selected, and a selected person in Group B is less likely to be truly suited (63.2% against 85.7%). Rule 3 gives equal selection shares, but it selects 42.9% of the unsuited people in Group B against 20% in Group A, and fewer than half of those selected in Group B are suited. Each fix improves one measure and moves another.

The reason is that the groups here differ in how many people are truly suited (60% against 30%) and the tool is not perfect. Research papers by Kleinberg, Mullainathan and Raghavan and by Chouldechova show that when real rates differ between groups and a tool makes mistakes, some of these goals generally cannot all hold at once. You will work through this with code in AI Ethics and Responsible Use: Advanced. For now, the practical point is that choosing a fairness goal means choosing which cost to accept, and that choice must be made openly.

Documented case: a metric conflict

Documented case:

COMPAS and ProPublica, 2016. ProPublica, a US newsroom, published "Machine Bias", an
analysis of a risk-score tool made by Northpointe and used in Broward County, Florida.
Reported by ProPublica: among defendants who did not reoffend, Black defendants were labelled
higher risk more often than white defendants (44.9% against 23.5%, as ProPublica reported).
Northpointe disputed ProPublica's methodology and defended the tool's accuracy.
Established: the numbers ProPublica published, and that the company disputed the analysis.
Disputed: whether the tool is "biased". The answer depends on which fairness definition you use.
Not covered: later versions of the tool, or other jurisdictions.

This is the standard example of two reasonable fairness definitions pointing in different directions. It does not show that either side was simply right. It shows why a fairness claim should always say which measure it uses.

Where reasonable people disagree

Should a hiring or admissions tool aim to select the same share of each group (demographic parity)?

Both views want a fairer result. People in different countries and organisations weigh equal shares, equal chances for qualified people, and individual treatment differently, and rules about using group information also differ. A sensible team picks the measure that fits the task, records why, and checks other measures as well.

Why one goal does not fit every task

Equal error rates also have limits. They can only be checked when you have a trustworthy "truth" for each person, and in hiring and admissions that truth (job performance, course completion) is often measured badly or comes from past decisions. Equal error rates may then equalise mistakes against a flawed yardstick.

Proxy variables

Page 5 introduced the proxy: a feature that stands in for something else. In a fairness review, ask four questions about each feature a tool uses.

1. Does it relate to the task for a reason you can explain (not only because it happens to predict)?
2. Could it stand in for a group, such as postal area for neighbourhood or a career gap for caring duties?
3. If you removed it, would the tool treat some group differently (test with and without)?
4. Is there a more direct measure of what you really want, such as a short work sample instead of the name of a university?

Small groups and uncertain measurements

Two practical problems make fairness numbers shaky.

Small groups. With few people, one case moves a rate a lot. Toy data:

A group has 12 applicants and the tool advances 9 of them: 9 / 12 = 75.0%.
If one more person had been advanced:   10 / 12 = 83.3%.
If one fewer person had been advanced:   8 / 12 = 66.7%.
One case moves the rate by about 8 points, so a gap of 8 points means little here.

The honest approach is to report the counts, say that the group is small, and avoid strong claims in either direction. Do not refuse to look at small groups. You can combine evidence over a longer time, read the individual cases, or ask a reviewer to look at every case.

Uncertain measurements. A fairness check compares the tool's decision with some "truth". If that truth is itself uncertain (a manager's rating, a judge's decision, a teacher's grade), the check inherits the uncertainty. Ask who produced the label, how consistent they were, and whether their judgement could differ by group. If it could, say that the numbers measure agreement with past judgement, not real ability.

A hypothetical hiring-screening scenario

Invented scenario:

Harbourline Foods, a mid-sized food distributor, receives about 600 applications for 20
warehouse-supervisor jobs. Jonas, the HR lead in Berlin, wants a tool to cut the first read
from three weeks to three days. The vendor's tool does four things automatically:
  1. Rejects any CV with a work gap longer than 6 months.
  2. Rejects applicants who live more than 40 km from the site.
  3. Scores the rest on keywords that match the job advert.
  4. Sends the top 60 to interview. Nobody reads the rejected 540 applications.

Apply the decision pattern to the workflow, not just the tool. Who is affected? Applicants, who never see the rule; current staff; the company, which may lose strong candidates. The benefit is speed. The harms fall mostly on applicants who are rejected without a human read. Steps 1 and 2 are exclusionary: they remove people before any measure of skill, and they rely on two proxies (a gap may stand in for caring duties or illness; distance for neighbourhood or transport). Step 3 may favour people who write like the advert. Step 4 makes every false negative invisible, because nobody checks the 540. Even if each step is lawful and well meant, together they can leave out capable people for reasons that have little to do with the job.

Legal note (as of 2026-10-12; not legal advice): Rules for automated hiring tools differ by place and change. Two examples taken from official pages. In New York City, an employer or employment agency may not use an automated employment decision tool unless it has had a bias audit within the past year, audit information is public, and candidates were given notice (the city's Department of Consumer and Worker Protection, DCWP, enforces it, and enforcement began on 2023-07-05). In the EU, the Commission lists employment among the areas where AI systems can be high-risk, with duties such as risk management, data quality, logging, human oversight and accuracy for the provider; whether a particular tool is high-risk depends on the legal text, and the Commission's timeline lists 2027-12-02 for the high-risk rules for those areas after the 2026 amendment. Neither example says what the law is where you live. Check the current official text for your country and sector.

A fairer alternative

A fairer workflow does not need to drop the tool. It changes what the tool is used for and where people look.

Worksheet, filled in:

1. SITUATION       Cut the first read from 3 weeks to 3 days for about 600 applications.
2. WHO IS AFFECTED Applicants (no say), the hiring team, the supervisors who will work with hires.
3. BENEFITS/HARMS  Benefit: faster answers. Harm: capable people rejected for a gap or an
                   address; nobody sees a wrong rejection.
4. OPTIONS         A. Vendor workflow as shown (auto-reject on gap and distance).
                   B. Remove the gap and distance rules. Tool only sorts; a person reads all.
                   C. Replace the CV filter with a 20-minute work sample in a text-first format
                      and a person-readable score; tool never rejects.
                   D. No tool: add two temporary readers.
5. EVIDENCE        Known: nothing about our own applicants. Test: run the tool on last year's
                   600 and compare with the people who were hired and did well. Check results
                   for applicants with gaps, distant addresses and different CV styles.
6. DECISION        Option C with B's safeguard. The tool may rank; a person reads every application
                   in the lowest band quickly; 50 of them get a full read each month. Stop if two
                   suited people are found in the lowest band in one month.
7. ACCOUNTABLE     Jonas. The hiring manager signs off each shortlist.
8. REVIEW DATE     2027-01-31: compare shortlist makeup, time saved, and any complaints.

Notice what changed. Exclusion was replaced by ranking, a more direct measure (the work sample) replaced two proxies, a person can see the false negatives, and there is an owner and a date. The workflow is slower than the vendor's design. That cost is real, and it is the trade-off to state. A hiring manager who needs 20 people in a week may reasonably choose differently, and a good decision says what was given up.

Where this shows up at work

A school admissions office asks whether a ranking tool helps pupils who study in a second language. A bank team has to decide how many honest customers it is willing to block to stop fraud. A freelancer building a form for a client finds that a postal-code field rejects valid rural addresses. A hospital administrator is asked to review a vendor's claim that a tool is "fair". In each case the useful move is the same: ask which fairness goal is meant, who bears each kind of error, and what evidence exists for each group.

Common mistakes

• Saying "the tool is fair" without naming the measure. It can pass one and fail another. Fix: say which definition, which groups, which counts.
• Treating equal treatment as enough. One rule for everyone can still block some people. Fix: ask whether everyone can actually use and succeed under the rule.
• Ignoring who pays for each kind of error. Fix: write what a false positive and a false negative cost, and who bears each.
• Removing a sensitive field and calling it done. Proxies can carry the same information. Fix: test with and without, and compare groups.
• Making strong claims from tiny groups. Fix: report counts, say the group is small, and use other evidence such as reading cases.

Try it yourself

Toy data: a scholarship tool screens applicants and "truly ready" was decided after the first term.

Group C: 200 applicants, 50 truly ready. Tool selected 60: 45 ready, 15 not ready.
Group D: 100 applicants, 20 truly ready. Tool selected 18: 12 ready, 6 not ready.

1. Easy. For this scholarship tool, say in words what a false positive and a false negative are, and who bears the cost of each.
2. Medium. For each group, work out three numbers: the share of applicants selected, the share of truly ready applicants who were selected, and the share of not-ready applicants who were selected. Say which fairness goals from this page the tool meets.
3. Hard. Propose a fairer alternative to this rule: "reject any applicant who did not submit the application on the website between 08:00 and 17:00". Write the eight-step worksheet in short form with at least three options (one without AI), name one trade-off, and describe how two reasonable people could disagree about the best option.

Answers

Easy. A false positive is a not-ready applicant who is selected; the cost falls on the scholarship fund and, if the person struggles, on that person. A false negative is a ready applicant who is not selected; the applicant loses the scholarship, and may not know why. A strong answer also notes that in this task the false negative is usually the more serious harm for the individual, while a funder with a small budget may care more about the false positive, and that this is a judgement. Pass: both mistakes are defined in the context, with who pays. Weak: the answer only repeats the definitions.

Medium. Group C: selected 60 of 200 = 30%; ready who were selected 45 of 50 = 90%; not-ready who were selected 15 of 150 = 10%. Group D: selected 18 of 100 = 18%; ready who were selected 12 of 20 = 60%; not-ready who were selected 6 of 80 = 7.5%. The selection shares differ (30% against 18%), so demographic parity is not met. Equal opportunity is not met either: ready applicants in Group D are selected much less often (60% against 90%). The false-positive rates differ a little (10% against 7.5%), so equalised odds is not met. Among those selected, the share who are ready is 75% in C and 66.7% in D, which are fairly close. Pass: the three numbers per group are right, and the answer says that several goals fail while one is roughly close. Weak: calling the tool "fair" or "unfair" with no measure named, or ignoring that D has only 20 ready applicants, so a few cases change the rates a lot.

Hard. A strong answer sees that the time-window rule is a proxy: it can exclude people who work in those hours, have shared devices or poor connections, or live in another time zone. Sample worksheet: (1) Select ready applicants fairly. (2) Applicants, the selection panel, the fund. (3) Benefit: fewer late files to handle; harm: capable people excluded by time and connection. (4) A: keep the rule. B: accept applications for a week and read all. C: tool ranks, panel reads all in the lowest band. D: no AI, two panel members read everything. (5) Test the rule on last year's files: how many ready applicants were excluded? (6) Option B with C's ranking only as a helper; stop if the panel finds that the ranking misses ready applicants. (7) The scholarship officer. (8) A date after the next round. Trade-off: reading everything takes more panel time. Disagreement: one person may say that a clear deadline is itself fair and cheap; another may say the deadline's real purpose is cost, and cost should not decide who gets a chance. Pass: the proxy is named, there are three or more options with one non-AI, a trade-off is stated, and both views are fair. Weak: a verdict with no options, evidence or owner.

Look back at page 5 and use its bias checklist on this scholarship tool: name three possible sources of bias. (Possible answers: "truly ready" was decided after the first term, so it may reflect who received support during that term; people who could not apply at all are missing; reviewers may read only the selected list.) Then compare Haruka's reading-support tool from page 5, which had the same two mistakes: which one would you rather it make, and why?

Summary

• Equal treatment and equal outcomes are different goals. A single rule can still block people who start from a different place, as accessibility shows.
• Every screening decision has false positives and false negatives. Their costs differ by task, and a value judgement about who pays belongs in the decision.
• Fairness has several definitions (equal selection shares, equal chances for qualified people, equal error rates, equal meaning of "selected"). With different real rates between groups and an imperfect tool, they generally cannot all hold at once.
• Question every proxy, report counts for small groups, and remember that a fairness check inherits the uncertainty of its labels.
• A fairer workflow replaces exclusion with ranking and human reading, uses direct measures, and names an owner and a review date.

Next: Trust, Verification and Automation Bias: When to Rely on AI Output turns from fairness to truth: how to tell a checked claim from a plausible one, and why a person who reviews the output can still miss errors.