Algorithmic hiring tools and bias: what employers should check before they buy

Algorithmic hiring tools promise faster, more consistent screening, and some vendors promise less bias than human recruiters. The evidence shows they can deliver on consistency and can also reproduce bias at scale. The difference usually lies in what the employer checked before buying.

Why algorithmic hiring tools can be biased

Most algorithmic hiring tools learn from data, and data reflects the past. The best-known example remains the recruiting engine Amazon built from 2014 to rank CVs. According to a 2018 Reuters investigation, it was trained on a decade of applications that came mostly from men, and it learned to penalise CVs containing the word "women's" and to downgrade graduates of two all-women's colleges. Engineers neutralised those terms but could not guarantee the system would not find other proxies, and the project was abandoned.^[1] No one had designed the tool to discriminate; it simply learned what past hiring looked like.

Newer technology has not removed the problem. Wilson and Caliskan audited text-embedding models of the kind used to match CVs to job descriptions, across nine occupations and more than 500 CVs. The models favoured White-associated names in 85.1% of cases and female-associated names in only 11.1%, with Black men disadvantaged most consistently.^[2] Proxies such as names, postcodes, gaps in employment and the institutions people attended can all carry group information into a model that never sees a protected characteristic directly.

The other side of the argument

Bias is not unique to algorithms. Human judgement is also shaped by past patterns and first impressions, and the selection research consistently favours structure: in Sackett and colleagues' 2022 re-analysis, structured interviews, which apply the same questions and scoring to everyone, were the top-ranked predictor of job performance.^[3] A well-designed algorithm applies the same criteria to every applicant, can be tested for adverse impact in a way that a recruiter's intuition cannot, and can be corrected and retested. The fair comparison is therefore not algorithm versus perfect judgement, but a specific tool, with its evidence, against the process it replaces.

What vendor claims do and do not tell you

Raghavan and colleagues reviewed the public claims of 19 vendors of algorithmic pre-employment assessments. Only five published any validation information. Vendors that made concrete claims about bias generally focused on outcome parity, typically the US "four-fifths rule", under which a selection rate for any group below 80% of the highest group's rate signals possible adverse impact. Their usual approach was to build a model, test for group differences, remove features that produced them and test again.^[4]

The authors identified two problems with relying on this. Passing a four-fifths check says nothing about whether the tool predicts job performance, or whether it predicts equally well for different groups. And the check depends on representative data and on the employer's own applicant pool, which may differ from the vendor's.^[4] A tool can be compliant on one dataset and still unfair or simply invalid in your context. In the UK the relevant legal test is in any case different: the Equality Act 2010 asks whether a practice puts a protected group at a particular disadvantage and, if so, whether it is a proportionate means of achieving a legitimate aim.^[5] A ratio is evidence; it is not a defence on its own.

Questions to ask before you buy

The Society for Industrial and Organizational Psychology's recommendations put it plainly: AI-based assessments should meet the same standards as any other employment test, covering accuracy in predicting job outcomes, consistency, fairness, sound operational use and full documentation.^[6] In practice that translates into five sets of questions.

Validity. What does the tool measure, and what evidence links it to performance in roles like yours? Ask for the criterion used (supervisor ratings, retention, training outcomes), the sample size and the correlation obtained, and whether the evidence comes from independent or peer-reviewed work.

Adverse impact. What are the score or selection-rate differences by sex, ethnicity, age and disability? Were they measured on samples like your applicants? How often is the model re-audited, and will the vendor support monitoring on your own data?

Training data and features. What was the model trained on, and does it use data such as video, voice, social media or CV text that may act as a proxy for protected characteristics? Does the model change after deployment?

Explainability. Can the vendor explain, in terms a candidate would understand, why one person scored higher than another? If not, you will struggle to answer a challenge or to meet data protection transparency duties.

Human review and adjustments. Is the tool designed to support a decision or to make it? Can reviewers see enough to overrule it? How are disabled candidates accommodated? The UK Information Commissioner's audit of AI recruitment tools recommended that recruiters should not progress or reject candidates on the basis of AI-generated scores alone.^[7]

Buying with evidence

None of this means avoiding algorithmic tools. It means applying the same scrutiny you would to any psychometric test: job analysis first, evidence of validity and fairness for your use, transparency for candidates, and a human who owns the decision. Tools that cannot answer these questions are not ready for high-stakes hiring, whatever the efficiency gains. For the standards behind each question, see our article on what makes a psychometric test valid. The same questions apply to established methods such as those in our assessment catalogue.