25,500 resume screenings. One researcher ran that experiment across 10 different AI models, swapping only minor identity and demographic variables on identical work histories. An independent AI auditor reviewed the results and flagged a 45% bias rate.
The number is striking. But the mechanism behind it is what should concern anyone using AI in hiring.
"Silent Bias" Is Harder to Catch Than the Obvious Kind
The models weren't generating overtly discriminatory language. Instead, the researcher describes what happened as "silent bias": when a model downgraded a candidate based on a demographic signal, it didn't say so. It generated professional-sounding justifications - critiques of communication style, leadership presence, strategic thinking - that appeared in evaluations of some candidates and not others reviewing the same underlying credentials.
This is the worst-case version of AI bias from an auditing perspective. If a model explicitly flagged a demographic characteristic, you'd catch it in a basic review. When it fabricates plausible critique to explain a score difference, you need systematic testing to find the pattern. A recruiter reviewing the output sees coherent reasoning. They don't see that the same resume with a different name scored 12 points higher.
The study tested 10 models, suggesting this isn't one vendor's problem. Bias at this scale across this many systems points to a training data issue: models trained on historical hiring data absorb the discriminatory patterns embedded in that data and reproduce them when scoring new candidates.
What AI Hiring Tools Don't Tell You
AI resume screening is already embedded in hiring workflows at companies that would never describe themselves as using "experimental" technology. The pitch is efficiency - process hundreds of applications in minutes, surface top candidates before humans spend time reviewing. That efficiency argument is real. The question is what gets lost in exchange.
Screening that happens before a human sees a resume is where bias causes maximum damage. There's no interview transcript to review, no decision memo, no trail that reveals why a qualified candidate was never advanced. The model's output is the filter, and if that filter has a 45% flagged-bias rate, a meaningful share of your candidate pool may be getting scored on factors that have nothing to do with their ability to do the job.
Fine-tuning (the process of training a model further on specific domain data to sharpen its performance on a narrow task) doesn't automatically fix this. If the training data reflects historical hiring patterns from companies with discriminatory practices, fine-tuning on that data can compress and amplify the underlying biases rather than correct them.
The Test Most HR Teams Haven't Run
The most direct response to these findings is also the simplest: test your screening tools. Run identical resumes with names and demographic signals changed and compare scores. This isn't a research project - it takes under an hour and requires no technical expertise. If scores diverge significantly across demographically different presentations of the same credentials, you have a problem worth escalating before regulators or plaintiffs find it first.
Several U.S. cities and states have passed or proposed legislation requiring bias audits for AI hiring tools. New York City's Local Law 144 requires annual third-party bias audits for automated employment decision tools. Illinois and Maryland have notice and transparency requirements. These regulations are moving faster than most HR teams have updated their vendor contracts.
The 25,500-screening study is one data point. It won't be the last.