---
title: 2 Motivation
---

# 2 Motivation

## 2.1 What the reviewers told us, and what the data confirms

The pilot with the Belastingdienst produced two reviewer questionnaires, one for the Box 3 vacancy and one for an earlier vacancy. Eight reviewers answered. The answers were linked to the evaluation data of the Box 3 vacancy – 198 answers from 33 candidates – so that each piece of feedback could be tested against what the scores actually show \[1\]. The questionnaire itself is very negative: on the question whether they would recommend human-plus-AI assessment to other selection committees, both Box 3 reviewers answered 0 out of 10 and the earlier vacancy's reviewers ranged from 0 to 7.

The feedback groups into four topics. The most consistent one is efficiency: seven of eight reviewers say the process is not more efficient, because judging each answer separately takes too much time. This cannot be checked in the evaluation data, since review time per answer is not logged; it is a process finding, not a scoring finding. The second topic is consistency: reviewers notice that contradictions between a candidate's answers are not weighed when every answer is scored on its own, and they want one picture of the candidate. The data confirms the mechanism behind this: a candidate's levels vary less across their six answers under the AI than under the reviewers – a spread ratio of 0.79 – so the AI makes candidates look more uniform than reviewers judge them.

The third topic is the CV: reviewers say that answers alone are not enough and that candidates refer to a CV the reviewer cannot see. Only two of 198 answers mention a CV explicitly, so the data gives weak evidence here, but the competency with the lowest reviewer average is the one that asks for qualifications and years of experience – material a CV would normally supply. The fourth topic is scoring: the AI should be stricter and sharper and look at what was actually answered. This is partly supported.

The overall bias is small (the AI's average level is 0.05 above the reviewers' and the candidate ranking correlates at 0.88), but the AI compresses the scale: 69.7 % of its scores sit on level 3 against 54.5 % of the reviewers' scores, and it uses the ends of the scale on 6.6 % of answers where reviewers use them on 16.9 %.

![Figure 2 – Reviewer feedback by topic next to what the Box 3 data confirms.](../media/image3.png)

*Figure 2 – Reviewer feedback by topic next to what the Box 3 data confirms.*

The analysis draws one conclusion that this concept takes as its starting point: better scores alone will not change the feedback. What reviewers ask for is a compact and consistent overview of the whole candidate with less effort – a product question before it is a scoring question \[1\].

## 2.2 What the customer cases ask for

The objectives and priorities overview \[2\] defines four product objectives – Dynamic Profile, Decision Support, Dashboard and Talent Pool – and ties their acceptance conditions to concrete customer cases. Read together, the cases ask for a change in the unit of assessment. IND needs decision quality under pressure: criterion-level evidence per candidate, compared on one screen. NVWA needs explainable selection and talent continuity: evidence that stays usable after the vacancy is closed. TNO wants evidence before CV review, which presupposes a structured requirements profile and evidence collected against it rather than a document to read.

SZW needs skills visibility and assessor consistency, and the TalentLead cases need skills mobility and talent reuse – a candidate's evidence matched against another requirements profile without a new assessment. None of these is served by a score per vacancy and nothing else. All of them are served by a normalised evidence profile per candidate that can be interpreted against several requirements profiles.

The ontology note \[16\] states the same two motivations from the system side: recommendations, talent pooling and matching across positions and organisations need a reusable candidate profile, and complementary evidence types let organisations balance assessment depth, candidate effort and assessor time against application volume.

## 2.3 What the redesign has to deliver

Four outcomes follow. Flexible matching: reusable evidence that can be compared with several requirements profiles instead of a re-assessment per vacancy. A broader picture: background, specific examples and demonstrated work next to the behavioural items. Proportionate effort: assessment depth that follows the position and the recruitment stage, and the rule that nothing a candidate already recorded is asked again. Traceable assessment: every conclusion cites its requirement, the scoring guide used and the original evidence. The rest of this report is organised around these four outcomes and the constraints under which they must be reached.

![Figure 3 – The two needs and the four intended outcomes.](../media/image4.png)

*Figure 3 – The two needs and the four intended outcomes.*