Game-based assessment has moved from novelty to procurement shortlist. The engagement benefits are real and reasonably well evidenced. The measurement claims deserve more scrutiny than they usually get.
What is genuinely different
The strongest argument for game-based assessment is not measurement, it is experience. Completion rates are typically higher, drop-off lower, and candidate reactions more positive than for conventional tests. For high-volume early-careers hiring, where a proportion of candidates abandon the process partway, that is a material commercial effect.
There is also a plausible anxiety argument. Test anxiety adds noise to scores, and a task that feels less like an exam may produce a cleaner measure for anxious candidates. The evidence is suggestive rather than settled, but the mechanism is credible.
And some formats capture things conventional items cannot — how someone approaches a problem, how they respond to feedback, how strategy changes as difficulty rises. Process data, not just outcomes.
The measurement question
Here the ground gets softer. A game measures whatever it measures, and the burden is on the supplier to show what that is.
Two questions cut through most marketing. First: what does this correlate with? If a game claims to measure numerical reasoning, it should correlate substantially with established numerical reasoning measures. If it correlates weakly, it is measuring something else — which may still be useful, but is not what the label says.
Second: what is the norm group, and how big is it? Game-based assessments are newer, which means norms are often smaller, younger and less representative than for established tests. A percentile is a comparison, and a comparison against 200 graduates from one sector is a different claim from one against 20,000.
Ask both questions early. Suppliers who have done the work will answer readily.
Where the evidence is thin
Fairness evidence lags. There is reasonable evidence on candidate reactions and growing evidence on criterion validity, but far less on differential item functioning and adverse impact — partly because these instruments are young and partly because the analyses are harder when the task is dynamic.
Accessibility is a genuine and under-discussed problem. Games often depend on speed, fine motor control, colour discrimination or sustained visual attention in ways conventional tests do not, and the adjustment routes are less obvious. A speeded game may not be adjustable by extra time at all without changing what it measures.
Practice effects also deserve attention. Games are more engaging, which means candidates are more likely to have played something similar. If prior exposure improves scores, the test partly measures exposure.
How to evaluate one
Treat it exactly as you would any other instrument, and do not let the format earn it a discount.
Ask for the technical documentation. Ask what construct it claims, what it correlates with, and on what sample. Ask when the norms were collected and from whom. Ask what fairness analyses have been run. Ask what happens for a candidate who needs an adjustment.
Game-based assessment is not a gimmick — the engagement benefits are real and worth having. But engagement is a reason to prefer one valid instrument over another. It is not a substitute for validity.
- Engagement and completion benefits are real and reasonably well evidenced
- Ask what it correlates with, and what the norm group consists of
- Fairness and accessibility evidence lags well behind the marketing
- Judge it as an instrument — the format earns no discount on the usual questions