Walsenuk Stop Babysitting Agents Framework

// When should you build a Context Engine for your agents?

Use this skill whenever you are building or evaluating an agentic coding system and find yourself repeatedly correcting agents, triggering every job manually, or supplying context that the agent should already know. Also use it when designing the infrastructure layer that sits behind background or headless agents.

// What do you need before designing a Context Engine?

• Current agent setup descriptionrequired
  How agents are currently spawned and what context (if any) they receive today — e.g. Claude CLI, Cursor, Codex, raw MCP connections.
• Code repository or codebase detailsrequired
  The repo(s) the agents work against — languages, patterns in use (e.g. factory pattern), key services.
• Systems of record inventoryrequired
  All SaaS apps, communication platforms, docs stores, and data sources that engineers actually rely on (Slack, Jira, GitHub, Zendesk, etc.).
• Team / org size
  Approximate number of engineers; signals how urgently data governance and permissioning matter.
• Target agent task
  A concrete task you want a background or agentic system to handle without babysitting (e.g. 'implement a new first-class Zendesk integration').

// What principles govern the Stop Babysitting Agents framework?

// How do you build a Context Engine step by step?

1. 1

   Diagnose your current position on the Context Ladder

   Identify which stage your team occupies: (a) Fancy Autocomplete — no agentic loops; (b) You Are the Context Engine — you trigger every job and correct every output; (c) Curated Context Layer — static files like CLAUDE.md, agents.md, a docs repo; (d) Context Engine — runtime, multi-surface, personalised retrieval; (e) Fully Autonomous Agents. Be honest. Most teams are at (b) or (c). This determines your next investment.

2. 2

   Audit your systems of record and identify all context surfaces

   List every place useful engineering context lives: GitHub (PRs, commit history, code patterns), Slack / Teams (decisions, CTO overrides, tribal knowledge), Jira / Linear (tickets, priorities), internal docs, runbooks, SaaS integrations. Do not assume your static repo covers this. It does not cover runtime signals or conversational decisions.

3. 3

   Build or configure a Social Graph for your engineering org

   Generate a graph where nodes are engineers and edges represent collaboration signals: who reviews whose PRs, who authors in which services, who pairs with whom. Node size can encode shipping volume. This graph is the pivot point — when a query arrives, the engine uses it to scope retrieval to the right codebases, people, and history for that specific engineer. Point an automated tool at your code repo to generate this procedurally; do not maintain it by hand.

4. 4

   Replace naive RAG with exhaustive, multi-surface retrieval

   Do not stand up a single vector store and call it done. Naive RAG triggers Satisfaction of Search — the agent stops at the first plausible chunk. Instead, build retrieval that: (1) constructs a structured research query from the agent's intent, (2) fans out across all systems of record in parallel, (3) runs exhaustively until no new relevant signals remain, (4) reasons across results before returning anything to the agent.

5. 5

   Implement conflict resolution logic

   When two sources contradict — e.g. code in main vs. a Slack thread — apply authority-weighting rules: recency, role (CTO > peer comment), and canonicity (official doc > off-hand message). Surface the conflict and the resolution to the agent with citations. Never silently pick one. Log conflicts for human review; they are architectural signals.

6. 6

   Enforce data governance and permission-scoped responses

   Carry auth context (OAuth model) through every retrieval call. Private Slack DMs, restricted channels, and confidential data must never surface to a requester who lacks permission, even if the agent query would benefit. If your org is 20+ people, this is not optional. Design the engine to return only what the requesting identity is authorised to see.

7. 7

   Compress and token-optimise the context packet before injecting into the agent

   The engine's output to the agent is not a raw dump. Reason across all retrieved surfaces, strip redundancy, resolve conflicts, and produce a small, high-signal research packet. Do not attempt to fill a million-token context window — large context windows do not help agents reason; they cause them to fail. Smaller, curated packets produce better plans and cheaper runs.

8. 8

   Structure agent execution as: Plan with Engine → Execute → Review with Engine

   Use the Context Engine at two critical junctures: (1) before execution to produce a correct, org-aware plan; (2) at code review to evaluate the output against real patterns, past decisions, and current truth. Execution in the middle runs with the plan as its harness. This three-phase loop is what produces PRs that get 'nitpick and merge' rather than 'this would break the entire system'.

9. 9

   Do not cache context answers

   Caching a correct answer is equivalent to writing docs — the moment you write it, it begins to decay. A cached answer to 'what is our Zendesk pattern?' is probably wrong 24 hours later because something changed. Optimise for latency through better retrieval architecture, not answer caching.

10. 10

    Extend the Context Engine to non-agent surfaces

    The same engine that serves background agents should also serve: Ask Engineering Slack channels (auto-detect questions, score confidence, respond automatically), ticket enrichment, incident triage, and human engineers asking ad-hoc questions. You get compounding leverage from a single well-built engine.

// What does the Context Engine framework look like in practice?

// What mistakes should you avoid when building a Context Engine?

• Treating naive RAG over a docs store as a Context Engine — it is not; it triggers Satisfaction of Search and produces confident but wrong outputs.
• Connecting more MCPs and assuming the agent will figure it out — MCPs are pipes that provide access, not understanding; the agent still does not know what it does not know.
• Believing a large context window (1M tokens) solves the problem — agents cannot reason effectively over that volume; a token-optimised small packet outperforms a stuffed context window every time.
• Hiding conflicts when two sources disagree and letting the agent pick — the agent will pick wrong; conflicts must be resolved explicitly with authority-weighting before the packet is delivered.
• Caching context answers for latency — cached answers decay almost immediately in active codebases; a cached correct answer becomes a confident lie by the next day.
• Skipping the Social Graph and treating all engineers and queries as identical — personalised relevance requires knowing who is asking, what they own, and who they work with.
• Ignoring data governance until the engine is already ingesting Slack and Teams — private conversations must be permission-scoped from day one, not retrofitted.
• Assuming the static curated context layer (CLAUDE.md, agents.md, a docs folder) is sufficient — static content has no runtime signals, goes stale, and requires manual maintenance.

// What are the key terms in the Stop Babysitting Agents framework?

Context Engine

The machine-layer that replaces the human engineer as the supplier of org-specific context to agents. It ingests all systems of record, builds a social graph, performs exhaustive multi-surface retrieval at runtime, resolves conflicts, enforces permissions, and returns a token-optimised research packet to the agent.

You Are the Context Engine

The stage of AI adoption where the human engineer manually supplies all context — pointing at files, correcting mistakes, triggering every job — because no machine layer exists to do it.

Curated Context Layer

The intermediate stage where teams maintain static files (CLAUDE.md, agents.md, internal wikis) that agents can read. Better than nothing but limited by staleness and absence of runtime signals.

Satisfaction of Search

The phenomenon (borrowed from radiology) where an agent stops searching the moment it finds a plausible answer, missing the actual canonical pattern, root cause, or correct implementation. The primary failure mode of naive RAG.

Social Graph

A graph of engineers as nodes and collaboration signals (PR reviews, co-authorship, service ownership) as edges. Used as the pivot point for personalised, scoped retrieval — 'you are this engineer, you own these codebases, here is what matters to you'.

Research Packet

The token-optimised, conflict-resolved, permission-scoped output the Context Engine delivers to an agent before execution. Contains exactly the org-specific facts the agent needs to plan and act correctly — nothing more.

Exhaustive Retrieval

A retrieval strategy that fans out across all relevant data surfaces and runs until no new relevant signals remain, rather than stopping at the first plausible result. The antidote to Satisfaction of Search.

Conflict Resolution

The process of detecting when two sources contradict each other, applying authority-weighting rules (role, recency, canonicity), and surfacing the settled truth — with citations — to the agent rather than silently picking one source.

Token Optimisation

The compression step where the Context Engine reasons across all retrieved data and returns only the minimum high-signal content the agent needs. Prevents agents from receiving bloated context that degrades reasoning quality and inflates cost.

Doom Loop

The babysitting cycle where an engineer repeatedly corrects an agent's output — pointing at files, explaining org patterns, re-running prompts — because the agent lacks the context to get it right autonomously.

Context Ladder

The progression of AI adoption maturity from fancy autocomplete → you are the context engine → curated context layer → context engine → fully autonomous background agents.

// FREQUENTLY ASKED QUESTIONS