Hetzel Agent Team Composition Framework

// When should you use the Hetzel Agent Team Composition Framework?

Use this skill whenever a team is deciding who should own agentic AI development — especially when an existing ML or data science team has been handed agent-building responsibility by default, or when a new AI initiative is being staffed from scratch.

// What inputs do you need before applying the Hetzel framework?

• Organisation Typerequired
  Is the organisation a Traditional Enterprise (existing ML/DS teams, delegated mandate) or an AI Native (built around agents from the start)?
• Agent Use Caserequired
  What problem is the agent meant to solve? Who are the end users?
• Current Team Compositionrequired
  Who is currently assigned to build the agent — data scientists, ML engineers, product engineers, subject matter experts, PMs, or some mix?
• Agent Architecture Complexity
  Is this a single agent or a distributed multi-agent system (supervisor + sub-agents on different infrastructure)?
• Fine-Tuning Requirement
  Does the use case require fine-tuning an open-source model, or is the team working entirely with pre-built LLM APIs?

// What are the core principles behind the Hetzel Agent Team Composition Framework?

// How do you apply the Hetzel framework step by step?

1. 1

   Classify the organisation type

   Determine whether this is a Traditional Enterprise (ML/DS team handed the agent mandate top-down) or an AI Native (small, cross-functional, agile team built around agents). The classification shapes the default risk: Traditional Enterprise teams tend to over-index on ML metrics and under-include non-technical experts; AI Natives risk under-engineering rigour and guardrails.

2. 2

   Audit the current team composition for coverage gaps

   Map current team members against three personas: (1) Data Scientists / ML Engineers, (2) Product / Application / Systems Engineers, (3) Non-Technical Domain Experts or Subject Matter Experts. Identify which personas are missing or marginalised. A team staffed entirely by ML engineers is a warning sign.

3. 3

   Assign Data Scientists / ML Engineers their agent-specific role

   Their role is NOT to train the model — that is already done. Assign them to: (a) act as the 'adult in the room' on LLM risk and statistical literacy, (b) validate LLM-as-judge evals against labelled datasets using recall, precision, and F1, and (c) lead fine-tuning of open-source models if the use case genuinely requires it. Redirect them away from obsessing over traditional ML metrics as the primary eval signal.

4. 4

   Assign Product / Application / Systems Engineers their agent-specific role

   These engineers implement requirements into the product, manage the systems and infrastructure where agents execute (especially critical for distributed multi-agent architectures with supervisor and sub-agents on different compute), and build the eval and observability pipelines that close the feedback loop between production and experimentation.

5. 5

   Assign Non-Technical Domain Experts their agent-specific role

   These people have the most proximity to the problem. Give them meaningful control over prompt and context engineering — the primary lever for changing agent behaviour. Also deploy them in human annotation workflows: they should review agent traces and label whether the agent performed well or poorly, and critically, explain WHY. Do not treat this as optional or cosmetic.

6. 6

   Define the eval and observability pipeline jointly

   Evals (pre-production experimentation) and observability (post-production monitoring) are the two pillars of agent quality. The team must agree on what 'good' looks like functionally — not just technically. Use production data to continuously expand the offline evaluation dataset. Check whether LLM-as-judge evals are converging toward or diverging from human agreement over time.

7. 7

   Pressure-test the team against the 'Proximity to the Problem' principle

   Ask: does the team have at least one person who deeply understands what the end agent is actually meant to solve? If the team is entirely engineers with no domain expert involved, the agent will lack the contextual grounding needed to be relevant. Fix this before building, not after.

// What does the Hetzel framework look like in real-world scenarios?

// What mistakes should you avoid when staffing an agentic AI team?

• Handing agentic development entirely to ML or data science teams because 'it has AI in the name' — this is the most common Traditional Enterprise mistake and leads to teams optimising for the wrong metrics.
• Obsessing over precision, recall, and F1 as the primary eval signals for agents — these are technical metrics suited to a two-box ML pipeline, not to the broad functional surface area of agentic behaviour.
• Ignoring non-technical domain experts or treating their input as cosmetic — these people hold the most proximity to the problem and are the primary contributors to prompt/context engineering and human annotation quality.
• Treating fine-tuning as the default approach — fine-tuning open-source models is rare and should only be pursued when the use case genuinely demands it; most agent behaviour is changed via context engineering, not retraining.
• Skipping the observability pipeline post-production — confidence in an agent built in experimentation does not transfer automatically to production; real usage confronts the agent with scenarios that evals did not anticipate.
• Letting LLM-as-judge evals run unchecked without validating them against human-labelled ground truth — judges are just prompts and models; they can drift from human agreement without a self-check mechanism.
• Treating agents as just another predictive model and applying the traditional cross-validation and AB-testing dance — the pipeline is entirely different once the model is already built.

// What are the key terms in the Hetzel Agent Team Composition Framework?

Agent Quality

The discipline of ensuring an agent performs correctly both before and after production, comprising two pillars: evals and observability.

Evals

Evaluations performed during experimentation and development to build confidence in an agent's execution before it is pushed to production.

Agent Observability

Monitoring an agent's behaviour after it is in production to maintain confidence in its execution as it encounters real users and real usage.

Proximity to the Problem

The degree to which a team member understands what the end agent is actually meant to solve. Higher proximity — typically found in domain experts and SMEs — leads to better context engineering and annotation quality.

Context Engineering

The primary lever for changing agent behaviour: adjusting the prompts, context, and inputs fed to a pre-built LLM rather than retraining or feature engineering.

LLM-as-Judge

Using a language model to evaluate the outputs of an agent as part of the eval process. Requires validation against labelled datasets to ensure the judge itself is trustworthy.

Human Annotation Workflow

A structured process in which domain experts review agent traces and label whether the agent performed well or poorly — and explain why — to generate grounded training and evaluation signal.

Agent Trace

A logged record of an agent's execution steps, decisions, and outputs that can be reviewed by technical or non-technical evaluators.

Traditional Enterprise

An organisation that approaches agentic development by delegating it to an existing ML or data science platform team, typically because generative AI is categorised as an 'AI problem'.

AI Native

An organisation that built its entire offering around agents from the start, typically characterised by small, cross-functional, agile teams with high proximity to the problem and no legacy ML platform.

Functional Performance

Evaluation of whether an agent actually accomplishes its intended purpose for real users — as distinct from technical performance metrics like precision, recall, and F1.

Distributed Agent / Supervisor + Sub-Agents

A multi-agent architecture in which a supervisor agent orchestrates multiple child or sub-agents running on different infrastructure, calling different systems — a complex systems engineering problem.

// FREQUENTLY ASKED QUESTIONS