Structured hiring

How to run structured interviews at volume

From job-based questions and anchored scales to batch invitations, calibrated review and candidate follow-up.

By MSZ Hire Editorial Team · 7 min read

When you are filling one role, a well-run conversation can be enough. When you are hiring dozens of call centre agents, warehouse workers or support workers at a time, small inconsistencies multiply. Different interviewers ask different questions, "good" means different things to different people, and some candidates wait weeks for an answer. Structured interviews fix most of that, and they hold up best when you plan them as a process, not just a list of questions.

This guide walks through that process. It applies whether a person or software asks the questions.

#Why structure matters

A structured interview asks every candidate for a role the same job-related questions, under the same conditions, and rates every answer against the same standards. The US Office of Personnel Management describes structured interviews as using "rules for eliciting, observing, and evaluating responses", and says that interviews with more structure show higher validity and rater reliability.

A widely cited 2022 review of selection research by Sackett, Zhang, Berry and Lievens, published in the Journal of Applied Psychology, ranked structured interviews among the strongest predictors of job performance of the methods it compared. Structure also makes decisions easier to explain, because you can show what each candidate was asked and how each answer was rated.

#Step 1: Start from the job

Before writing questions, list the three to five things that separate people who do the job well from people who struggle. Use the job description, but also ask a supervisor and someone who does the job today.

Good areas are specific and observable: handling an upset customer, following a safety procedure under time pressure, explaining a technical trade-off. Weak areas are personality labels such as energetic or a good fit, which invite reviewers to fill in the gaps with their own impressions.

Then group them into a small number of scoring areas. MSZ Hire uses four for every role (technical competency, problem solving, communication and practical experience), so scores are comparable across roles.

#Step 2: Write questions that produce evidence

Each question should give the candidate a fair chance to show one or two of your areas. Two formats work well:

  • Past behaviour: "Tell me about a time you had to calm down an upset customer. What did you do, and what happened?"
  • Situational: "A pallet arrives damaged ten minutes before the end of your shift. Walk me through what you'd do."

Keep each question to one idea, in plain language, answerable without insider knowledge of your company. Read it aloud: if it's hard to say, it's hard to answer.

Avoid questions that touch protected grounds such as age, family status, religion, disability or place of origin. The Ontario Human Rights Commission's hiring guidance is a good checklist for what to leave out.

#Step 3: Anchor your rating scales

This is the step most teams skip, and the one that makes the biggest difference at volume. For each area, write what a 1, 3 and 5 answer sounds like, using examples from the actual job. Our post on structured interview scoring rubrics has worked examples.

Then set the weights: how much each area counts toward the overall score. Decide them before you see any answers, write them down, and don't change them mid-campaign. If you must change them, start a new version so earlier and later candidates aren't scored differently.

#Step 4: Deliver it the same way to everyone

Consistency covers more than the questions:

  • The same instructions, in the invitation and at the start of the interview.
  • The same time. Give each question a fixed time limit that is long enough for a full answer.
  • The same scoring, applied to what the candidate said, not to how they look or sound.
  • The same rules about help. Decide what isn't allowed (notes, another person, a second screen) and say so up front.

Consistency doesn't mean rigidity. Some candidates need an accommodation, such as no video, typed answers, more time or an interview with a person. In Ontario, the accessibility standards require employers to tell applicants selected for an assessment that accommodations are available on request, and to consult them and arrange a suitable accommodation if they ask (O. Reg. 191/11, s. 23). Build that route into the process from day one; see our checklist for employers in Canada.

#Step 5: Invite in batches with clear deadlines

At volume, the calendar becomes the hard part. A few habits help:

  • Invite in waves sized to what your reviewers can handle in a week, rather than all at once.
  • Give a realistic start-by date, often a week, so candidates can choose a quiet time. Send a new link to anyone who asks for more time.
  • Tell candidates exactly what they need: the length, the device and browser, a quiet room, and how to ask for an accommodation.
  • Track starts and completions by wave. Low completion is usually a sign that the invitation is unclear, the deadline is too short or the technology is getting in the way.

#Step 6: Review at speed without cutting corners

Volume tempts teams to sort by score and stop reading. Resist it.

  1. Calibrate first. Have every reviewer rate the same five to ten interviews on their own, then compare. Talk through the differences until everyone reads the anchors the same way.
  2. Read the evidence. For each area, check that the quote behind the rating actually shows what the rating says.
  3. Double-check the edges. Have a second reviewer look at borderline interviews, interviews that ended early, and interviews with integrity signals.
  4. Treat integrity signals as questions, not answers. Leaving the window or a face check that didn't run can have innocent causes, such as a dropped connection or assistive technology. Watch the recording before you draw conclusions.
  5. Record the decision and the reason. A one-line reason now saves hours if a candidate, a manager or a regulator asks later.

#Step 7: Close the loop with every candidate

Candidates who interview remember how they were treated, and many of them are also your customers. Tell everyone what happens next and when.

In Ontario, employers with 25 or more employees must tell applicants they interviewed for a publicly advertised job, within 45 days of the last interview, whether a hiring decision has been made (O. Reg. 476/24, s. 5). Even where that rule doesn't apply, a short, timely update is the simplest way to improve candidate experience. Our post on candidate experience in AI interviews has more ideas.

#Step 8: Learn from your own data

After each campaign, look at:

  • Completion: how many invited candidates started and finished, and where they dropped off.
  • Time to decision: from invitation to a recorded decision.
  • Score spread: if almost everyone scores the same, your questions or anchors aren't separating candidates.
  • Reviewer agreement: on the interviews two people reviewed, how often did they agree?
  • Candidate feedback: what candidates said about the experience.
  • Outcomes over time: with legal advice, check whether results differ across groups of candidates, and whether the people you hired are doing well.

Our post on benchmarking and bias audits explains how teams test their process.

#A one-page plan

Step Decide or do Done
1. The job Three to five specific areas that predict success ☐
2. Questions One or two per area, plain language, no protected grounds ☐
3. Scoring Anchors for 1, 3 and 5 per area; weights written down ☐
4. Delivery Same instructions, time and rules; accommodation route ☐
5. Invitations Waves sized to review capacity; realistic deadlines ☐
6. Review Calibration, evidence first, second reviewer on the edges, reasons recorded ☐
7. Follow-up Every candidate told what happens next ☐
8. Learning Completion, time, score spread, agreement, feedback ☐

#Where MSZ Hire fits

MSZ Hire is built around this process. You paste a job description and get a draft interview of 4 to 10 questions with anchored scoring for four areas, which you edit and publish as a locked version. Every candidate hears the same questions, read aloud and shown on screen, in a random order, with a time limit for each. Answers are scored against your weights with quotes as evidence, and a person on your team records the decision. Candidates can request accommodations before they start. See how to create a role and how to invite candidates to get going.

Questions and answers

What makes an interview structured?

Every candidate for the role is asked the same job-related questions under the same conditions, and every answer is rated against the same defined standards. The US Office of Personnel Management describes structured interviews as using rules for eliciting, observing and evaluating responses.

How many questions should a structured interview have?

Enough to cover the few things that matter most in the job, usually one or two questions per area. In MSZ Hire, an interview has 4 to 10 questions, and a 15-minute interview allows up to 7.

Should every candidate get the questions in the same order?

The questions and the scoring should be the same. Changing the order for each candidate makes it harder to pass answers on to later candidates, and it doesn't change what is being measured. MSZ Hire randomizes the order for each candidate.

Who should review interviews when there are hundreds?

A small group of trained reviewers who read the evidence, not just the score. Calibrate them on a shared sample first, and have a second person look at borderline and flagged interviews.

Sources

Primary sources, checked on 18 November 2026. Dates are those shown by each source.

  1. Structured Interviews (opens in a new tab)US Office of Personnel Management. Accessed 18 November 2026.
  2. Revisiting meta-analytic estimates of validity in personnel selection (Sackett, Zhang, Berry and Lievens), Journal of Applied Psychology, 107(11) (opens in a new tab)American Psychological Association. 2022.
  3. Human Rights at Work (third edition), IV-5: Interviewing and making hiring decisions (opens in a new tab)Ontario Human Rights Commission. 2008.
  4. O. Reg. 191/11: Integrated Accessibility Standards (s. 23) (opens in a new tab)Government of Ontario, e-Laws. Version in force from 30 March 2026.
  5. O. Reg. 476/24: Rules and Exemptions re Job Postings (s. 5) (opens in a new tab)Government of Ontario, e-Laws. Version in force from 1 January 2026.

Read next on the blog

See how MSZ Hire handles it

Structured voice interviews with advisory, evidence-backed scoring, accommodations on request and data hosted in Canada. A person makes every decision.