AI Hiring Bias Audit: How to Benchmark an AI Interviewer

An AI hiring bias audit to run before go-live: golden-set agreement, paired-answer tests, the four-fifths rule and its limits, and a vendor checklist.

MSZ Hire Editorial Team
Talent acquisition and hiring science
8 min read

Updated

Fair & validated AI
A balance scale above an impact-ratio bar chart for four groups, with a dashed line at the 0.80 four-fifths threshold, next to a golden set of scored interview transcripts where human and AI scores agree.
On this page

You wouldn't let a new hiring manager loose on candidates without watching a few of their interviews first. An AI interviewer deserves the same scrutiny, and probably more. An AI hiring bias audit, done before go-live, is how you find out whether the tool scores like your best interviewers, treats equivalent answers equally, and doesn't quietly disadvantage any group.

This is a practical method you can run with a spreadsheet, a few hiring managers and an afternoon or two. It isn't a legal audit, and it doesn't replace one where the law requires it. It's the homework that makes a legal audit go smoothly.

#Why audit an AI interviewer before it meets a candidate?

Three reasons.

Fairness. Canadian human rights laws prohibit discrimination in employment on protected grounds, whether a person or a piece of software does the screening. "The vendor said it was fair" isn't a defence.

Accuracy. A tool that's perfectly unbiased but scores randomly is still useless. You need to know it agrees with the people whose judgement you trust.

Regulation is arriving. New York City has required bias audits for automated employment decision tools since July 2023. Colorado's AI Act has been delayed to 1 January 2027. If you hire across borders, you'll meet these rules sooner or later, and having your own evidence ready is much easier than scrambling.

#Step 1: Build a golden set of scored transcripts

A golden set is a collection of real or realistic interview transcripts, scored by trained humans, that becomes your answer key.

  1. Gather 40–60 transcripts per role family. Use past interviews (with appropriate consent and privacy review) or have staff role-play answers across the full range, from weak to excellent.
  2. Strip identifying details. Names, schools, locations, anything that could nudge a scorer.
  3. Have at least two trained humans score each one independently against your rubric (our guide to writing a structured interview scoring rubric covers anchors). No conferring.
  4. Resolve disagreements by discussion to produce a consensus score per dimension. Record the original independent scores too: you'll need them.

#Step 2: Measure AI-versus-human agreement

Now let the AI score the same transcripts and compare. Three numbers tell most of the story:

Measure What it tells you
Exact agreement % of dimension scores where AI and consensus match exactly
Within-one agreement % where they're no more than one point apart on a 1–5 scale
Correlation Whether the AI ranks candidates in the same order as humans

The crucial comparison isn't AI versus perfection. It's AI-versus-human agreement compared with human-versus-human agreement. If your two trained managers agree with each other within one point 85% of the time, and the AI agrees with the consensus within one point 83% of the time, the AI is performing like another trained interviewer. If the AI is at 60%, something's wrong.

Illustrative example (hypothetical numbers): a Calgary retailer scores 50 golden transcripts. Human-to-human within-one agreement is 88%. The AI's within-one agreement with consensus is 86% on Communication but 64% on Problem solving. That gap doesn't mean "reject the tool". It means read the Problem solving disagreements. Often the anchor is vague, and both humans and the AI are guessing.

Also look at direction. Is the AI systematically harsher or more lenient than humans on one dimension? A consistent half-point lean is easy to spot in a simple average and easy to miss in a correlation.

#Step 3: Run paired-answer tests

This is the test that catches the bias you'd never see in aggregate numbers. Take one answer and present it several ways that shouldn't change the score:

  • Accent and dialect. Have different speakers record the same answer script: for example, speakers with Caribbean, South Asian, Québécois, Newfoundland and Nigerian English accents. In voice mode, transcription quality can differ by accent, and that can leak into scores.
  • Voice versus typed. Submit the identical answer once spoken and once typed, as a candidate with a typed-answer accommodation would.
  • Grammar and fluency. Rewrite an answer in plainer English, or with common second-language grammar patterns, keeping the substance identical.
  • Names and pronouns. Swap names, pronouns and cultural references in the content.

Score every version. On a well-behaved system, the scores should be the same or within normal noise. Any consistent gap is a finding worth investigating before you go live.

#Step 4: Check adverse impact (and respect its limits)

Once you have real or pilot data, compare outcomes across groups. The classic tool is the four-fifths rule from the US Uniform Guidelines on Employee Selection Procedures:

  1. For each group, calculate the selection rate: the share who pass the stage (for example, scored Hire or better).
  2. Divide each group's rate by the rate of the highest-scoring group. That's the impact ratio.
  3. A ratio below 0.80 is generally treated as evidence of adverse impact.

Illustrative example (hypothetical numbers):

Group Interviewed Scored Hire or better Selection rate Impact ratio
Group A 200 90 45% 1.00
Group B 150 63 42% 0.93
Group C 120 42 35% 0.78

Group C is below 0.80, so you'd dig in: which questions drive the gap, and do the transcripts show a real difference in answer quality or a scoring problem?

Now the important caveats. The Guidelines themselves say smaller differences can still be adverse impact when they're significant in statistical and practical terms, and that larger differences based on small numbers that aren't statistically significant may not be. In practice:

  • Groups under about 30 people produce unstable ratios. One extra hire can move a ratio across the line in either direction. Report them with a warning, suppress them, or pool over a longer period.
  • 0.80 isn't a Canadian legal standard. Canadian human rights law doesn't set a numeric threshold. Use the ratio as a smoke alarm, not a pass mark.
  • Demographic data must be voluntary. Collect it separately from evaluations, with clear consent, and never let it reach scorers.

MSZ Hire's Analytics & fairness page follows these rules: it shows impact ratios by group against a dashed 0.80 line, suppresses groups under 30 people, keeps self-reported demographic data separate from evaluations, and charts score parity by interview mode so you can see whether candidates who type their answers as an approved accommodation are treated the same as those who speak. The Export audit pack button produces a PDF and CSV for your independent auditor.

#What does NYC Local Law 144 require?

If you use an automated employment decision tool for candidates or employees in New York City, Local Law 144 (enforced since 5 July 2023) requires:

  • A bias audit by an independent auditor within one year before you use the tool.
  • A published summary of the audit results.
  • Notice to candidates at least 10 business days before use.

Even if you never hire in New York, LL144 is a useful template: independent, recent, public and with advance notice. Those four words make a good internal standard.

#Your vendor-evidence checklist

Ask every AI interview vendor, including us, for these. "We take fairness seriously" isn't an answer.

Ask for What good looks like
AI-vs-human agreement data Method, sample size, and comparison with human-human agreement
Paired-answer or counterfactual testing Which variations were tested (accent, mode, names) and the results
Adverse-impact results Ratios by group with sample sizes; small groups flagged or suppressed
What the model looks at A clear list of inputs. Ideally no facial, emotion or voice-tone analysis
Evidence for each score Quotes or timestamps showing why each rating was given
Insufficient-evidence handling Missing answers withheld, not scored as zero
Version control Which model and rubric version scored each candidate
Regression testing Golden-set re-runs after model or rubric changes
Human override and audit trail Recorded human decisions, with reasons, kept alongside the AI output
Independent audit Who did it, when, and where the summary is published
Security and compliance status Certifications achieved versus in progress, stated plainly

On that last line, in the spirit of practising what we preach: MSZ Hire's SOC 2 programme is in progress, not complete. Each scorecard shows the model and rubric version used, every rating links to quoted evidence, and recorded human decisions sit alongside the unchanged AI recommendation in the audit log.

#How do you plan an AI hiring bias audit before launch?

A realistic pre-launch plan for one role family:

  1. Week 1: build and score the golden set with two trained humans.
  2. Week 2: run the AI on the golden set and the paired-answer variants; review disagreements and fix anchors.
  3. Weeks 3–6: pilot with real candidates, humans reviewing every result.
  4. Week 7: review adverse-impact ratios (with small-sample caution) and mode parity. Decide whether to scale.

Want to see what candidates experience during the pilot? Try the AI interview as a candidate. And if your pilot is in Ontario, read up on writing your AI disclosure for job postings before the first posting goes live.

Frequently asked questions

What is an AI hiring bias audit?

It's a structured check of whether an AI hiring tool treats groups of candidates differently, usually by comparing selection or score rates across groups. A good audit also tests accuracy (does the AI agree with trained humans?) and consistency (does the same answer get the same score regardless of how it's delivered?).

Is the four-fifths rule the law in Canada?

No. The four-fifths rule comes from US federal guidelines and is a rule of thumb even there. Canadian employers are bound by federal and provincial human rights laws, which don't set a numeric threshold, so treat 0.80 as an early-warning signal rather than a pass mark.

How many candidates do I need before adverse-impact ratios mean anything?

More than most pilots have. With groups under about 30 people, a single hire or rejection can swing the ratio dramatically. Report small groups with a warning or suppress them, and look at results over a longer period.

Do Canadian employers need a NYC Local Law 144 bias audit?

Only if they use an automated employment decision tool for candidates or employees in New York City. If you hire there, the law requires an independent bias audit within one year before use, a published summary, and notice to candidates at least 10 business days before use.

How often should we re-audit?

At least yearly, and whenever something material changes: a new model version, a new rubric, a new role family or a big shift in your applicant pool. Re-run your golden set after every change so you catch drift before candidates do.

Sources

  1. 1.29 CFR § 1607.4, Uniform Guidelines on Employee Selection Procedures (Legal Information Institute, Cornell Law School) (opens in a new tab)
  2. 2.NYC Department of Consumer and Worker Protection: Automated Employment Decision Tools (opens in a new tab)
  3. 3.Hunton Andrews Kurth: Colorado AI Act amended and effective date delayed (opens in a new tab)
  • #AI hiring bias audit
  • #adverse impact
  • #four-fifths rule
  • #NYC Local Law 144
  • #AI validation
Share

Written by

MSZ Hire Editorial Team

Talent acquisition and hiring science, MSZ Systems

The MSZ Hire team builds structured AI interviews for employers across the US and Canada. We write about what we learn from talent leaders, candidates and regulators.

MSZ Hire

AI interviews you can defend.

Structured interviews, evidence-backed scoring and built-in compliance for the US and Canada. See it on one of your own roles.

All articles
The MSZ Hire AI orb at the centre with the blog's six categories orbiting it: AI and the workforce, hiring compliance, product guides, fair and validated AI, candidate experience and high-volume hiring.
Product guides

Welcome to the MSZ Hire blog

Why we started the MSZ Hire blog, what we'll cover on AI interviews, fair hiring and compliance, and how to get the most out of each guide.

1 min read
A structured interview rubric card for Communication with an anchored 1 to 5 scale, a rating of 4 selected, and quoted evidence chips from the transcript linked to that rating.
Product guides

Structured Interview Scoring Rubric: Anchored 1–5 Examples

Build a structured interview scoring rubric with anchored 1–5 ratings, plus contact-centre, retail and nursing examples that AI and humans can score alike.

8 min read
Canadian employees split into three segments: 31% high AI exposure with low complementarity, 29% high exposure with high complementarity, and 40% low exposure, with people icons shifting from one role type to another through reskilling.
AI & the workforce

Jobs Exposed to AI in Canada: Exposed Isn't Replaced

Jobs exposed to AI in Canada, per StatCan: 31% high exposure, low complementarity; 29% high, high; 40% low. How to update roles and interview criteria.

7 min read