How Do PMs Shape Agent Team Composition?
For Product managers on AI teams · Based on Hetzel Agent Team Composition Framework
// TL;DR
As a product manager on an AI team, the Hetzel Agent Team Composition Framework gives you a mandate you may not realise you have. Because the model is already built and behaviour changes through context engineering — not retraining — your proximity to the problem and end users makes you central, not peripheral. This page shows how PMs contribute to context engineering, lead human annotation workflows, and drive the joint definition of functional performance. It also helps you spot when a team is dangerously engineer-only and needs domain grounding before building.
Where does a PM fit in agent development?
Closer to the centre than you might expect. The Hetzel framework's principle of proximity to the problem puts product managers and domain experts in a position of disproportionate value. The people closest to what the agent is meant to solve produce the best context engineering and human annotation — and PMs, who live between users and the build, sit exactly there.
This is a departure from traditional ML, where PMs often hand off to a model team and wait. With agents, the model is already built, so the work shifts to implementing, evaluating, and contextualising it. That work is deeply product work: defining what 'good' means for real users, shaping the inputs, and closing the loop with production behaviour.
How do I contribute to context engineering?
Context engineering — adjusting prompts, context, and inputs to a pre-built LLM — is the primary lever for changing agent behaviour. It's not retraining or feature engineering, so it doesn't require deep ML skills. It requires deep understanding of the problem and the user, which is your core competency.
Claim meaningful control here alongside domain experts. When the agent behaves wrongly, the first move isn't 'retrain the model' — it's 'change the context'. As a PM you can prototype prompt and context changes, test them against real user scenarios, and prioritise which behaviours matter most. This is where creative control has shifted toward domain knowledge, not just technical depth.
How do I run a human annotation workflow?
Human annotation is a structured process where people with proximity to the problem review agent traces — logged records of the agent's steps, decisions, and outputs — and label whether the agent performed well or poorly, explaining why. As a PM, you can own this workflow: recruit the right domain experts, define the labelling rubric grounded in user value, and ensure the 'why' is captured, not just a thumbs up or down.
Treat this as central, not cosmetic. It's a common pitfall to ignore or trivialise non-technical input — yet annotation quality directly determines evaluation quality and ultimately agent quality. The reasoning captured in annotation also feeds context engineering, creating a virtuous loop.
How do I define functional performance with the team?
Agent quality rests on two pillars: evals (pre-production) and observability (post-production). Your job is to drive the whole team toward a shared definition of what 'good' looks like functionally — does the agent actually accomplish its purpose for real users — rather than settling for technical metrics like precision, recall, and F1 alone.
Push back when engineers over-index on F1: those are technical metrics for a two-box ML pipeline, not the broad functional surface area of an agent. Instead, anchor evaluation in outcomes users care about: correctness, safety, and resolution. Then ensure production data continuously expands the offline eval dataset, because confidence from experimentation doesn't automatically survive real usage.
How do I spot a broken team before we build?
Run the framework's final pressure test: does the team include at least one person who deeply understands what the agent is meant to solve? If it's entirely engineers with no domain expert or PM involvement, the agent will lack contextual grounding and be irrelevant. Raise this early — fixing it before building is far cheaper than after.
Next step
This sprint, claim your three roles: prototype a context-engineering change to a real agent behaviour, stand up a human annotation workflow on recent agent traces, and lead a team session to define functional performance in user terms — not just F1.
// FREQUENTLY ASKED QUESTIONS
Do I need technical skills to do context engineering as a PM?
No. Context engineering adjusts prompts and inputs to a pre-built LLM — it's not retraining or feature engineering, so it doesn't require deep ML skills. It requires proximity to the problem and the user, which is your strength as a PM. The framework deliberately shifts creative control toward domain knowledge, opening this lever to product people, not just engineers.
How is my role different from a traditional ML product manager?
In traditional ML you often hand off to a model team and wait. With agents the model is already built, so the work is implementing, evaluating, and contextualising it — deeply product work. You gain direct levers: context engineering, human annotation, and defining functional performance. Your proximity to the problem makes you central to agent quality, not a downstream stakeholder.
What should I do if our agent team has no domain expert?
Raise it immediately as a pressure-test failure. An agent built entirely by engineers with no domain grounding will lack the context to be relevant. Either recruit a subject matter expert or, if you have deep proximity to the problem yourself, step into context engineering and human annotation. Fix this before building — the framework is explicit that it's far cheaper than fixing it after.