Should Your ML Team Own the Agent Project?
For Enterprise ML/data science leaders · Based on Hetzel Agent Team Composition Framework
// TL;DR
If your ML or data science team just inherited an agentic AI initiative top-down, the Hetzel Agent Team Composition Framework helps you avoid the isolation mistake. It shows why the model being already built shifts the highest-leverage work downstream of the API — to context engineering, guardrails, and observability — and reassigns your data scientists to what they're uniquely good at: guardrails and LLM-as-judge validation. Use it to bring in systems engineers and domain experts, resist defaulting to fine-tuning, and stand up the evals-plus-observability loop that moves impressive POCs into reliable production.
Why did your ML team get handed the agent project?
Because generative AI has 'AI' in the name, agent development in Traditional Enterprises is almost always delegated top-down to the existing ML or data science platform team. The Hetzel framework calls this the isolation mistake. Your team likely has strong Python and model-training skills but limited exposure to LLM APIs, distributed sub-agent orchestration, and — most importantly — the real user workflows the agent is meant to serve.
The fundamental shift is that the model is already built. Anthropic, OpenAI, and Mistral have already run the ingestion, training, cross-validation, and deployment pipeline and exposed it as an endpoint. The highest-leverage work now lives downstream of the API. Recreating a training workflow is wasted motion.
What should your data scientists actually do?
Don't ask them to 'own agents' generically, and don't ask them to write prompts or build distributed systems. Their unique value is threefold: being the adult in the room on LLM risk (reminding everyone the model is predicting token after token, not knowing anything), validating LLM-as-judge eval quality using labelled datasets and precision/recall/F1, and preventing the team from over-trusting outputs.
This is the guardrails role. It's where your existing statistical rigour compounds. When a domain expert or engineer builds an LLM-as-judge evaluator, your data scientists prove whether that judge actually agrees with human reviewers — catching eval drift before it silently corrupts your quality signal.
Who else needs to be on the team?
Two role types you probably lack. First, product and application engineers to own API integration, distributed infrastructure for supervisor-and-sub-agent orchestration, and the eval-plus-observability pipeline. This is a systems problem, not a statistics problem. Second, non-technical domain experts — the loan officers, underwriters, or clinicians closest to the problem — who should directly co-own the prompts and context, and act as primary human annotators of agent traces. They know the edge cases your engineers will never anticipate.
Should you fine-tune?
Almost certainly not by default. Fine-tuning an open-source model is rare and represents the single highest-leverage technical contribution when it's genuinely needed — but most use cases are solvable through context engineering alone. Before allocating significant data science resource to fine-tuning, confirm the problem cannot be solved by changing the inputs. Reserve fine-tuning as a deliberate, scoped assignment.
How do you know you're production-ready, not just POC-prolific?
Many enterprise teams are prolific at building generative AI POCs but never implement the eval and observability pipelines needed to ship safely. The tell is confidence that lives only in experimentation. Evals build confidence before production; observability maintains it once real users arrive. Close the loop by harvesting production traces into your offline eval dataset and having domain experts human-annotate them continuously, so LLM-judge alignment can be tracked and self-corrected.
Stop measuring the agent on precision, recall, and F1. Those metrics belong to your evaluators, not the agent's end-to-end functional performance. Have your domain experts define what the agent must do correctly for a real user, then let your data scientists validate the mechanism measuring it.
Next step: Run a team audit today. List every person on the agent project by role type, flag any missing role, reassign your data scientists to guardrails and LLM-judge validation, and schedule your first domain-expert prompt-engineering session this week.
// FREQUENTLY ASKED QUESTIONS
My ML team has no LLM API experience — can they still lead the agent project?
They can contribute, but they shouldn't lead it alone. The model is already built, so the leverage is downstream of the API. Assign your data scientists to guardrails and LLM-as-judge validation, then bring in systems engineers for orchestration and domain experts for context. Leadership should reflect that diversity, not sit inside the ML team by default.
How do I justify adding non-technical domain experts to an AI team?
Frame it around proximity to the problem. Domain experts understand real user intent and edge cases better than any engineer, which makes them the highest-leverage contributors to context engineering and human annotation. In agentic AI, behaviour changes by changing inputs — and the people closest to the problem write the best inputs.
Should I stop using precision, recall, and F1 entirely?
No — reposition them. Don't use them as the agent's primary success criteria; that's a category error since they measure classifiers, not full agent traces. Instead, use them to validate whether your LLM-as-judge evaluator agrees with human labels. That validation is precisely where your data scientists add durable value.