Skip to content
Assessment Guide
Back to Articles
Guide4 min read

Validity documentation when the vendor says AI certifies skills

Josh Bersin describes a new platform where AI agents observe work and certify skills. This guide looks at what the word certified does not tell you, and the four things buyers should ask for in the validity documentation before trusting a certificate.

By RedaktionenSource: Josh Bersin, joshbersin.com, 6 November 2026, on CodeSignal's launch of Agentic Skills Intelligence

The news in brief

On 6 November 2026, analyst Josh Bersin published a piece on joshbersin.com about a launch from CodeSignal called Agentic Skills Intelligence. Bersin describes it as a development agent that watches and analyses the work a person does, assesses their skills and coaches them along the way. According to the article, the platform draws on a benchmarking dataset of more than 20 million records, and it models not only skills but also the behaviours and work activities that feed into them. Evaluator agents observe the work and build a capability profile that keeps updating over time.

Bersin writes that, in his view, CodeSignal is the only platform using AI to certify skills rather than merely assess or build them. That is his opinion, and he discloses that his own company uses CodeSignal in its certification programme for HR professionals. He also names other players in the market, including HackerRank, SHL, Eightfold and Pluralsight.

The article matters to anyone buying tests and assessments because it shows where the market is heading. Assessment is moving from a bounded pre-hire test to continuous observation of the work itself, and the language is moving with it, from measured to certified.

What the word certified does not tell you

A certificate sounds like a verdict. In psychometric terms, though, a certification is only as good as the documentation behind it. The question a buyer needs to ask is not whether the platform certifies, but what the certificate has been shown to predict.

It helps to separate three things that are often blurred together. The first is consistency, meaning the same person gets a broadly similar result when measured again. The second is that the assessment really measures the skill it claims to measure and not something else, such as typing speed or familiarity with a particular tool. The third is that the result relates to something the organisation cares about, usually performance in the job. A large dataset says nothing about any of the three on its own. More than 20 million records is a statement about scale, not about validity.

Four things to ask for

When a vendor describes AI agents that observe work and infer skills, the documentation should at a minimum answer four questions.

How is each skill defined, and which observable behaviours count as evidence for it? If the agent models work activities, the documentation should state which activities are included and why those were chosen.

How stable is the assessment? If the same person is observed across two periods doing similar work, how closely do the two profiles match? A continuous profile that swings sharply from week to week is hard to use for decisions.

What is the relationship with job performance, and in which roles and sectors has it been studied? A relationship found among software developers cannot simply be carried over to customer service or sales. The documentation should say which groups the studies are based on.

How do results distribute across groups? When an assessment is built on observed work, it depends on what work the person has been given access to. Employees with fewer or less visible tasks may end up with a thinner profile, and that says nothing about their skills.

When assessment becomes continuous

Classic validity documentation describes a test taken at a set moment under known conditions. An agent that observes work on an ongoing basis breaks with that model and raises questions older manuals do not answer. What data is collected, and does the employee know? When is the profile updated, and can a single bad week drag it down? And who can see the profile when it is used for internal mobility or promotion?

These questions are as much about data protection and employee involvement as about psychometrics. In the EU, a system that evaluates employees for promotion or task allocation will also need to be checked against the AI Act's rules for high-risk systems once those apply.

What this means for buyers

Bersin's article is a recommendation from an analyst with a disclosed relationship to the vendor. That does not make it uninteresting, but it makes it a starting point for questions rather than documentation. The shift towards continuous, AI-driven skills assessment is likely to shape the whole market, including the vendors Bersin names and the many he does not.

The practical advice is simple. Ask for the technical manual or validation report before the word certified does its work on you. Check which skills are defined, which groups the relationships were studied in, and how the results distribute. If the vendor cannot provide that documentation, the certificate is a marketing claim, not a measurement result.

Kilde: https://joshbersin.com/2026/11/codesignal-agentic-skills-intelligence-builds-validated-skills-for-real/