A résumé screener can post strong aggregate accuracy and still reject qualified candidates from one group at twice the rate of another. Nothing in that headline number reveals it. You have to go looking, deliberately, with a written plan.
A bias audit is that plan. It is less a single test than a sequence of decisions you can defend later: what counts as harm, which data you trust, which definition of fairness fits the decision, and what happens when the numbers disagree. Work through the stages below in order. Each produces an artifact you will want when someone asks how you know the model is safe to ship.
1. Define the harm before you touch a metric
Metrics are meaningless until you know what you are measuring. Before opening a notebook, answer three questions in a short paragraph: what decision does the system make, who does it affect, and what does a genuinely bad outcome look like?
"Denying credit to someone who would have repaid it" is a harm you can count. "Prediction error" is not. The distinction tells you which mistakes to weight and which groups to compare.
Then name the sensitive attributes that matter in your domain — age, disability status, ethnicity, gender, language, geography — and note which ones you are barred from collecting or acting on where you operate. If you cannot collect a protected attribute, plan the workaround now: a voluntary self-identification field, an external validation sample, or a proxy analysis with stated limits.
Agree on your stop threshold in advance. A criterion chosen before the results arrive survives the pressure that comes after them.
2. Map every data source, including what's missing
Bias usually enters through data long before it appears in outputs. Build a one-page inventory for each source, and be specific about the following:
- Provenance and permission. Where did the records come from, and what were you allowed to do with them?
- Collection window and context. Data gathered during a policy change or a hiring freeze encodes that moment's quirks.
- Coverage gaps. Who is underrepresented, and who is absent entirely?
- Labels. Who labeled the data, under what instructions, and how often did annotators disagree?
- Missingness. Which fields are empty, for whom, and how did you fill them?
- Proxy variables. Postcodes, device types, school names, and job titles often carry the sensitive information you thought you removed.
That last item explains most stubborn cases. A model with no race or gender field can still sort people by group if a nearby column is doing the work. Test for it rather than assuming.
3. Choose fairness metrics that fit the harm
Several fairness definitions exist, and they do not all hold at once. Pick the one that matches the harm from step one, and write down why.
- Demographic parity: equal selection rates across groups. Useful when the decision allocates a scarce resource and you want representation to be even.
- Equal opportunity: equal true positive rates. Useful when missing a qualified person is the main harm.
- Equalized odds: equal true positive and false positive rates. Stricter, and often infeasible.
- Predictive parity or calibration: a given score means the same thing across groups. Useful when scores drive human decisions.
Because these conflict, expect a trade-off and document it: which metric is primary, which are monitored as secondary, and who signed off. In regulated areas such as hiring, lending, housing, or healthcare, involve a qualified lawyer or compliance specialist before finalizing the approach.
4. Test slices, not just averages
Aggregate performance hides subgroup failure. Report results for each group, then for combinations — older applicants with a disability, for instance, or women in one region. Intersectional slices are where compounding disadvantage shows up.
Small slices produce noisy numbers. Print the sample count beside every rate, and treat a difference on a slice of forty people as a reason to investigate, not a conclusion. Where you can, report intervals instead of single figures, and separate false positives from false negatives so you can see which error type is concentrated where.
Pre-register the slices you intend to examine. Choosing groups after seeing results invites cherry-picking, even unintentionally.
5. Mitigate, then measure the whole thing again
Fixes fall into three rough families: adjust the training data (reweighting, resampling, adding coverage), change the training objective (constraints or penalties that push toward your chosen metric), or adjust outputs after the fact (group thresholds, or a human review layer for borderline cases).
Whichever you choose, re-run the full audit rather than the one slice that failed. A correction aimed at one group often shifts error onto another, and a narrow fix can quietly damage overall performance. Record any accuracy you gave up, and let the business owner decide whether that trade is acceptable.
6. Write it down and set review triggers
An audit that lives in a chat thread is not an audit. Produce a short document covering intended use, data sources, metrics chosen, results by slice, known limitations, and mitigations applied. Name an owner. List the conditions that force another review: a new data source, a change in the population, drift in the inputs, a threshold adjustment, or a reported incident.
Monitoring matters more than the launch gate. Most bias problems surface months later, when the world moves and the model does not.
A checklist you can run this week
- Write the harm statement and set a stop threshold.
- Complete the data inventory, proxies included.
- Choose one primary fairness metric with a written rationale.
- Pre-register the slices you will test.
- Run the audit and record sample counts with every rate.
- Apply mitigations, then re-audit the full set.
- Publish the summary, assign an owner, and schedule the next review.
None of this requires exotic tooling. It requires deciding, in writing and in advance, what you will accept — then checking honestly whether the model delivers it.
Photo: KATRIN BOLOVTSOVA / Pexels
