How to Audit a Chaotic Coding-Agent Rollout
For Engineering leaders auditing agent chaos · Based on Lou Bichard Software Factory Primitives Framework
// TL;DR
Engineering leaders use the Software Factory Primitives Framework to audit a chaotic coding-agent rollout and pinpoint the real problem. When teams coordinate agents through GitHub and Linear, the signal-to-noise ratio collapses and no one knows when to intervene. The framework's four-primitive audit reveals that Runtime, Orchestration, and Triggers are usually fine — Coordination is the missing primitive. The fix is a purpose-built coordination layer with gated micro-steps and on-the-loop human oversight, not more tooling or more agents.
Why is our coding-agent rollout so chaotic?
If your organisation adopted coding agents and it's now chaos — humans don't know when to intervene, the signal-to-noise ratio is terrible, and PRs pile up — you're almost certainly hitting the reusing-human-tools antipattern. GitHub and Linear were designed for human coordination. Used as an agent coordination layer, they produce overwhelming noise that makes it nearly impossible to know when and where to step in.
The instinct is to blame the agents or add more tooling. The framework says: audit first. The problem is structural, and it's almost always the same missing primitive.
How do I audit what's actually broken?
Run the four-primitive audit. For each — Runtime, Orchestration, Triggers, Coordination — mark it solved, partial, or missing:
- Runtime ✓ — your agents have somewhere to run
- Orchestration ✓ — you can spin agents up and down
- Triggers ✓ — webhooks and PR events bring them online
- Coordination ✗ — there's no purpose-built layer for agents to hand off work, gate progress, and collaborate
Coordination is almost always the gap. Naming it explicitly stops your team from over-investing in runtime infrastructure that isn't the bottleneck.
Why won't more agents or better tools fix this?
Because more parallel agents with humans orchestrating each one is NOT a software factory — it's just more noise. And GitHub or Linear can't become the coordination layer no matter how you configure them; their purpose is human collaboration, so agent traffic drowns the signal.
The fix is a purpose-built coordination layer. Three viable form factors: a state machine or workflow graph defining SDLC micro-steps with gates; a CLI gateway agents query to validate stage completion; or durable execution for interruption-safe flows. Human visibility should come from that layer's state — showing exactly where to intervene — not from PR noise.
Underpinning all of it: decompose your coarse SDLC into explicit micro-steps with machine-checkable gates. Agents skip coarse boxes and agent sycophancy produces false completion signals, so gates that rely on self-report will keep failing at scale.
How do I keep my engineers on-the-loop without bottlenecking them?
Design for humans on-the-loop, not in-the-loop — able to see state and intervene when needed, but never required to drive each step. Build visibility into sub-agent activity (parent plus sub-agent status, task lists), but resist routing all coordination noise back through human-facing tools, or you'll recreate the exact chaos you're auditing.
As you push further toward autonomy, treat the security surface as a prerequisite. VM isolation is baseline; audit agent permissions, repo access scope, and compromise blast radius. Security done late blocks further automation.
Next step: Convene the team, run the four-primitive audit on one whiteboard, mark Coordination as the gap, and commission a coordination-layer prototype — a workflow graph or CLI gateway — instead of buying more tooling.
// FREQUENTLY ASKED QUESTIONS
How do I tell my team the problem is coordination, not the agents?
Run the four-primitive audit publicly: mark Runtime, Orchestration, and Triggers as solved, and Coordination as missing. It reframes the chaos from 'the agents are bad' to 'we never built a coordination layer.' This stops blame and redirects effort from swapping agents to building the purpose-built coordination layer that's actually the bottleneck.
Why can't we just configure Linear better to coordinate agents?
Because Linear was designed for human coordination — no configuration turns it into an agent coordination layer. Agent traffic produces overwhelming noise that buries the signal humans need to know when to intervene. You need a purpose-built layer (workflow graph, CLI gateway, or durable execution) whose state surfaces where intervention is needed, and let humans stay on-the-loop from there.
What's the difference between on-the-loop and in-the-loop oversight?
On-the-loop means engineers can see system state and intervene when needed but aren't required to drive each step. In-the-loop means a human must actively drive every stage — which is the bottleneck that prevents a real software factory. Design your coordination-layer UX to show where to intervene, not everything happening, so oversight stays lightweight and non-blocking.