---
title: 13 Open questions for the follow-up
---

# 13 Open questions for the follow-up

Four questions decide whether the biographical part can go into production. They are stated with the options and considerations we see today; none of them is decided here, and the third is explicitly a request to the legal review rather than a legal opinion.

## 13.1 A – A valid training and evaluation corpus for CV ingestion

Every biographical level rests on the ingestion step – extract, map, standardise – and its quality can only be claimed against a corpus that is representative, labelled and legally usable. What has to be labelled per CV are the span-level entries (experience, education, certificate, task phrases), their taxonomy concepts (occupation, task, skill, knowledge, ISCED level) and the normalised attributes (duration, recency, responsibility); agreement between two annotators per field is the first validity measure of the corpus itself. Four sources are candidates, and they differ in realism and in what they may be used for. Pilot CVs already in the platform are realistic but are personal data collected for a different purpose (see C).

Candidate-confirmed passport records from the new flow produce labels as a by-product of the confirmation step and are the most defensible source once consent covers it. Synthetic CVs generated from the taxonomy give coverage of rare occupations, languages and formats, which is useful for recall testing but not for a validity claim. Public or licensed corpora, such as ESCO-annotated job and CV datasets, need a check of their coverage of Dutch public-sector vocabulary. Whatever the mix, the corpus has to be stratified by language, sector, seniority, CV style and the groups we monitor, because the extraction-recall measure of section 9.6 cannot be computed per group otherwise. The decisions needed are which source may be used for which purpose (evaluation only versus tuning), the target size per stratum, and who annotates.

The proposal for the work stream is to start with evaluation-only use of confirmed passport records and a synthetic recall set, and to tune only once C is answered.

## 13.2 B – Mitigating bias in practice

The design-time controls are in the concept: blind entries; a rational key over tasks and skills rather than occupation similarity; proxies off by default; no total; knock-outs that filter instead of rank; separate scores to limit anchoring. What practice adds is measurement and intervention. The internal insights service computes, on every scored population, the group gap, the pass-rate difference and the balance point from the Bias Monitoring Framework \[14\] per requirement and per link type, the extraction recall and correction rate per group, and agreement with and without the CV score visible. The intervention repertoire is deliberately small and readable: down-weight or disable a link type, exclude a node, add a transferable link to the taxonomy, change an anchor – each a rule change with an audit entry and a re-measurement.

For the learned components, the extractor and the mapper, constrained or debiased representations are evaluated on performance and segregation risk together, as \[9\] recommends. What the concept does not yet give is thresholds (which gap triggers which action), a review cadence per customer and a sign-off for rule changes; the proposal is to set thresholds after the first vacancy in Sprint 4 rather than before, and to have rule changes reviewed by the AI team together with the customer's assessor and logged with their measured effect. One limit has to be stated plainly: group variables are only available where a customer may legally collect them.

Where they are not, the system can still measure how much proxy attributes influence levels, but it cannot measure group gaps, and the fairness panel must say so rather than show a reassuring blank.

## 13.3 C – Legal implications and GDPR constraints

This subsection lists the questions a legal review has to answer; it is not legal advice, and the AI team does not own these decisions. They are raised now because their answers shape the training corpus and the bias measurement.

Purpose and legal basis. Candidate data was collected to assess an application. Using it to train or tune extraction and scoring models is a further purpose that needs its own basis and a compatibility assessment (GDPR Art. 5(1)(b), 6(4)). The data processing agreements with the pilot customers have to be read for whether model improvement is covered; if it is not, the options are consent of the candidates, a documented legitimate-interest assessment, or an amendment of the agreement per customer. Evaluation-only use on a customer's own vacancy is easier to justify than cross-customer tuning.

Special categories and group variables. Bias measurement needs group variables, some of which are special categories (Art. 9) or sensitive in practice. Collecting them solely for fairness monitoring, separated from the records, aggregated and with minimum group sizes is the usual pattern, but it has to be in the processing agreement and the privacy notice, and some customers will set their own conditions – SZW's written privacy requirements and FG clearance \[2\] are an example. Automated decision-making and transparency.

The design keeps a human judgement per requirement and no automatic rejection; knock-outs used as filters come close to Art. 22 and require the reviewer step to remain a real one. Candidates have to be told that a CV is standardised and scored, how, and how to correct it – the confirmation step is also the Art. 13, 15 and 16 mechanism. AI Act.

Recruitment and selection systems are high-risk under Annex III; the documentation already prepared for ISO/IEC 42001 and Article 10 data governance has to cover the biographical scorer – data sets, representativeness, bias examination, logging, human oversight – and the six properties of section 3.3 map onto it. Retention and reuse. A passport reused for another position or organisation is a new purpose, and applicant-data retention periods conflict with a long-lived passport unless the candidate owns it and consents, which is TNO's ownership model and should be the product's.

The concrete asks for the legal work stream are five: read the pilot agreements for model-improvement and fairness-monitoring purposes; decide per corpus source whether evaluation and tuning are permitted; draft the candidate-facing text for the quick-fill and confirmation steps; define the collection and separation of group variables; and confirm the retention and consent model for passport reuse.

## 13.4 D – Validating validity and bias-proneness

Validation proceeds from the ingestion to the claim. Extraction validity is precision and recall of entries, concepts and attributes against the annotated corpus of A, per language and sector, with the candidate correction rate in production as the running proxy. Scoring consistency is the agreement between the biographical scorer and qualified reviewers who see only the standardised entries, with agreement between reviewers as the ceiling; the Box 3 indicator set \[1\] is extended to biographical levels. Added information asks whether passport levels add information next to question and probe levels – incremental agreement with the final human judgement, not correlation with hiring.

Predictive validity is only possible with performance criteria after hiring; the customer cases with a baseline, IND and TalentLead 1 \[2\], are the candidates for a first local study, and until then the validity claim stays at the method level of section 3.1 and is labelled as such – which is also what \[4\] recommends when local studies are feasible. Bias-proneness is measured as group gaps and balance points per requirement and link type, the sensitivity of levels to proxy attributes, and a comparison of the rationally keyed score with a naive baseline on the same blind entries; that difference is the measured value of the method.

The protocol is the Sprint 4 evaluation on at least one new vacancy with the current flow as comparison, with populations, outcomes and separate tests defined in the validation strategy of the second sprint week, and with results flowing into the internal insights service so that they are not a one-off report.