What's Missing From Your AI-Native Agent Team?

Why is proximity to the problem your biggest advantage?

In an AI-native startup, everyone codes, no one inherited a legacy AI ownership structure, and per-person proximity to the problem is high. That's exactly the setup the Hetzel framework rewards. Because the model is already built, the highest-leverage skill isn't retraining — it's context engineering, changing agent behaviour by editing prompts, context, and instructions. Your team is already close enough to the product to do this well.

So lean into it. Make sure the people who understand your users — including non-engineer domain experts like clinic staff or operations leads — are directly editing and owning the context seeded into your agents. They catch edge cases your engineers will never anticipate.

What role is your team almost certainly missing?

The guardrails role. When everyone on the team is an engineer with no formal data science background, no one is stress-testing the LLM's statistical limitations or validating whether your LLM-as-judge evals actually agree with human judgement. This is dangerous because LLM-as-judge is itself just a prompt and a model — it drifts, and without validation you won't notice until users complain.

You don't necessarily need a full-time hire. Bring in a fractional data scientist, or explicitly assign one engineer to own guardrails and eval validation. Their job: remind the team the model is just predicting tokens, prevent over-trust, and use precision/recall/F1 against labelled data to confirm the judge is aligned with humans.

How do you build eval quality from day one?

Start harvesting grounded data immediately. From your first couple hundred production interactions, build a labelled dataset so eval quality can be measured against human agreement. This is the raw material for the evals-plus-observability feedback loop: evals build confidence during experimentation, observability maintains it once real users arrive, and the loop closes when production traces flow back into your offline eval dataset.

Because you're small, you can wire this loop early and cheaply. Doing so is what separates a team that's prolific at POCs from one that reliably ships to production.

How should you split ownership as you grow?

Map responsibilities to roles the way the framework prescribes. Product and application engineers own API integration, distributed infrastructure for supervisor-and-sub-agent orchestration, and the eval and observability pipeline. Whoever holds the guardrails role owns LLM-as-judge validation and labelled dataset creation. Domain experts own prompts, context, human annotation, and defining what good agent behaviour looks like and why.

Resist the temptation to treat fine-tuning as a rite of passage. It's rare and should be a deliberate, scoped decision — most of what you need is achievable through context engineering. Confirm the problem can't be solved with prompts and context before you spend a sprint fine-tuning an open-source model.

What functional criteria should define success?

Don't let precision, recall, and F1 masquerade as your agent's success metrics — they measure classifiers and evaluators, not end-to-end agent behaviour. Instead, have your domain-proximate people define what the agent must actually do correctly for a real user, then have your guardrails owner validate the eval mechanism measuring it.

Next step: This sprint, name one person as your guardrails owner, pull a domain expert into a prompt-editing session, and start logging your first 200 production interactions into a labelled dataset for eval validation.