How to Build Evals for a Customer Support AI Agent
For AI engineers building customer support agents · Based on Hetzel Eval Maturity Phases Framework
// TL;DR
If you're an AI engineer building a customer support agent, the Hetzel Eval Maturity Phases Framework tells you exactly what to do at each stage. Start by having a support subject matter expert review 10–20 real queries with written justifications, derive failure modes from those justifications, build deterministic and LLM-as-judge scorers, populate your dataset with real production traces, and activate a flywheel that turns support failures into agent improvements. Use it when your support agent works in demos but you can't prove it's production-ready.
Why do support agents stall before production?
Most customer support agents work fine in a demo and then stall because the team has no structured way to prove quality. Informal manual review — 'it looks good' — isn't defensible when a hallucinated feature or a missed escalation becomes a reputational or compliance risk. The Hetzel Eval Maturity Phases Framework fixes this by giving you concrete actions at each of four maturity phases, all in service of one north star: agent quality.
How do you start when you have no eval process?
You're at Level 1: Just Getting Started. Select 10–20 representative customer queries and run them through the agent. Critically, a customer support subject matter expert — not the engineer who built it — reviews each output. For every output they record two things: a thumbs up or down, and a written justification. Examples: 'Thumbs down — agent hallucinated a feature that does not exist' or 'Thumbs up — correctly escalated to a human agent.'
The justification matters more than the verdict. It externalises the domain knowledge your SME holds about what good support looks like — knowledge you'll need to scale later. Never skip it.
How do you turn annotations into scoring functions?
At Level 2: Measuring to Manage, feed all those thumbs-down justifications into a coding assistant like Cursor, Claude Code, or Codex, and ask it to extract and categorise the failure modes. You'll get a structured list of the real ways your support agent fails: hallucinated features, wrong tone, failed escalations, incorrect account lookups.
For each failure mode, decide if it's deterministic. 'Too many tool calls' or 'exceeded token budget' can be a code-based scorer. Subjective failures like 'hallucinated a feature' need an LLM-as-judge — and you build that judge's prompt using the SME's actual justification language. Then eval the eval: label a ground truth set of outputs by hand and confirm the judge agrees with your human expert before trusting it at scale.
How do you make evals reflect real support traffic?
Stop testing only synthetic questions. Capture real production or UAT traces of actual customer conversations and add them to your dataset, including enough context to represent the account state at the time. Your dataset becomes a snapshot of production, not a hypothetical test suite.
Then activate the flywheel: capture production traces, surface failing conversations via human review or automated tooling, pull them into your offline eval dataset, rerun evals, and use the results to guide your next agent improvement. This is playing offense — every support failure becomes fuel for measurable improvement.
What about agents that look up account data or open tickets?
If your agent calls tools, distinguish read-only context-gathering (looking up an account) from CRUD operations (opening or closing a ticket). Never let eval runs write real tickets to production. Embed the relevant system state into the trace, use mock APIs for CRUD calls, and if your ticketing system supports version queries, replay the state as it existed when the trace was captured.
Next step
Locate your support agent on the maturity continuum today. If you've only been reviewing outputs informally, run your first structured Level 1 annotation session with a support SME this week — 15 queries, verdicts plus justifications — and let those justifications become the foundation of your entire eval system.
// FREQUENTLY ASKED QUESTIONS
Who should review my support agent's outputs at Level 1?
A customer support subject matter expert — not the engineer who built the agent. The SME holds domain knowledge about what quality support looks like, including tone, correct escalation, and accurate feature descriptions. Their written justifications become the raw material for deriving failure modes and building LLM-as-judge prompts that scale their expertise.
How do I catch hallucinated features in a support agent?
Hallucinated features are a subjective failure mode, so use an LLM-as-judge. Build the judge prompt from your SME's justification language (e.g. 'the agent described a feature that does not exist'), then eval the judge against a human-labelled ground truth set to confirm it aligns with expert judgement before scoring at scale.
How do I test ticket creation without spamming production?
Treat ticket creation as a CRUD-based tool call. Set up a mock ticketing API that accepts calls without writing real data, embed the relevant system state into the captured trace, and configure your eval task to read that embedded state. Never let eval runs open real tickets in your production system.