Engineers reached certification 50% faster than with human mentors, with less reliance on senior developers.
Back to all
Certifying Notch engineers' AI skills using Sovera AI™
How Notch used its own agentic platform to close the skills gap 50% faster between engineers who understand agentic AI in theory and those who can build with it in production.
CLIENT NAME
Notch
INDUSTRY
IT
TECHNOLOGY
Sovera AI™
SERVICES
Agentic AI Systems
DURATION
2025 - Ongoing
The Context
Notch is an AI engineering partner building agentic systems, custom AI models, and enterprise-grade digital products for organisations that need business-critical quality without the overhead. With 170+ engineers working as one AI-native delivery team, systematically certifying agentic AI skills and fluency across the organisation became a practical necessity.
The Challenge
Certifying a team of engineers quickly and at scale is hard to achieve when relying solely on human mentors. Upskilling engineers requires practice on real problems with feedback that explains reasoning rather than just marking answers right or wrong. And conventional training approaches don’t produce this at the scale required.
The Solution
Notch built a certification application on top of Sovera AI™, its agentic orchestration platform. The application presents engineers with AI-generated scenario-based tasks. Sovera AI™ provides the orchestration, memory, and evaluation logic underneath: acting as the evaluator, evaluating each step in real time, explaining what is right and wrong, and advising engineers on how to improve their scores. Completed tasks generate certificates tied to real capability milestones.
Project Goals
-
1Provide certification at scaleProvide every engineer with access to structured feedback and progression regardless of team capacity.
-
2Evaluation first, mentorship secondEvaluate each step immediately and explain the reasoning, instead of simply marking answers as correct or incorrect.
-
3Recognized certification levelsStructure the program around tasks that generate certificates upon completion and catalog certification levels within the team.
-
4Memory compounds across cohortsEach cohort should outperform the last as the system builds institutional knowledge, refines evaluation criteria, and builds on what has worked before.
-
5Validate Sovera AI™ across use casesProve Sovera AI™ can fulfill multiple roles, including generating certification tasks, evaluating, and offering guidance.
The Process
01
Outlining certification journey
Mapping milestones, skill requirements, and level-up criteria to create a structured path for agentic mentorship.
02
Designing the agent and UI
Specialized agents added for task generation, evaluation, feedback, and certification. Zero platform changes.
03
Build and evaluate the agents
Built and evaluated AI agents with three-tier memory for continuity, progress tracking, and accumulated cohort insights.
04
Build the system and integrate agents
Integrated intelligent agents and intuitive UI into Sovera AI™ to deliver and accelerate certification.
05
Deploy and monitor the agents
Monitor agent interactions, output quality, and user satisfaction. Feed insights back into agent improvement.
06
Monitoring performance
Tracking token usage, costs, and efficiency via telemetry to optimize the certification journey.
Play
Take a closer look at Sovera AI™, our agentic AI acceleration platform. How it’s built, what it runs on, and what it makes possible.
