How DevTools Founders Can Build Agent Skills That Replace Docs
For Developer tools founders and DevRel leads · Based on Klingen Coding Agent Skill Architecture Method
// TL;DR
If you're a developer tools founder or DevRel lead, the Klingen method helps you build coding-agent skills (CLAUDE.md, .clinerules) that guide agents through your product's integration patterns reliably. Instead of hoping agents read your 400-page docs correctly, you design progressive skill files with agent sitemaps, style rules, and search endpoints. You iterate using trace analysis and LLM-as-judge evals, then accelerate with auto-research loops. The result: users get correct integrations in fewer turns, and you get production signals showing what they actually need.
Why do coding agents keep getting my product's integration wrong?
Coding agents like Claude Code, Cursor, and Codex rely on pre-training knowledge that goes stale within months. If your SDK added a new tracing method last quarter, agents won't know about it. They'll hallucinate the old API, attempt it, fail, then spend extra turns self-correcting — if they correct at all.
The Klingen Coding Agent Skill Architecture Method solves this by treating the agent skill file as a product artefact. Instead of dumping your entire documentation into the agent's context, you create a structured instruction set that progressively discloses the right references at the right time.
What goes into a skill file for my developer tool?
Your skill file has two components: style rules and an agent sitemap.
Style rules govern behaviour: 'Ask the user whether their app is chat, batch, or RAG before recommending an instrumentation pattern.' 'Fetch the CLI help flag before assuming parameters exist.' 'If this skill is older than 30 days, alert the user.'
The agent sitemap is a structured index of your documentation URLs — not the docs themselves. This follows the Klingen method's core principle of Reference Over Duplication: embedding docs creates a local cache that goes stale, reproducing the exact problem you're solving.
For products with large doc surfaces (100+ pages), expose a search endpoint that accepts natural-language queries and returns relevant chunks. This reduces the turns the agent needs to find information and — critically — gives you production signals about what users are actually trying to do.
How do I measure whether my skill is actually working?
Start with a basic LLM-as-judge eval. Create a sample repository representing a realistic user application. Write 3–7 natural-language assertions: 'OpenAI instrumentation was added,' 'retrieval spans appear in trace,' 'no hallucinated API calls present.' Run these assertions against filesystem and trace state before and after skill execution.
Before automating, walk traces manually. Read the full execution record of what the agent did. Look for wandering behaviour, hallucinated method names, and missing spans. Each observation becomes a concrete rule or reference addition in the skill file.
Once your eval suite is running, use auto-research loops to generate skill improvement candidates. Define your target function with extreme precision — include every behaviour you want preserved, not just the primary success metric. Human-review every suggestion. Expect to accept about 50%.
What production signals should I track after launch?
Monitor your search endpoint queries to discover what users are actually asking agents to do — it's often different from what you assumed. Track trace data for hallucinated API calls, excessive self-correction turns, and missing instrumentation spans. These signals feed directly back into skill improvement.
The Klingen method's principle of Production Signals Over Assumptions means your skill evolves based on real usage, not design-time guesses.
What's the next step?
Audit your product's current coding-agent experience. Run a typical user request — like 'add observability to my project' — through a coding agent without any skill file, capture the trace, and document every failure. That trace is your baseline, and the Klingen method's 10-step workflow starts there.
// FREQUENTLY ASKED QUESTIONS
How long does it take to build a first coding agent skill for my developer tool?
A first usable skill can be built in 1–2 days. Start by auditing the pre-skill failure state (Step 1), then write style rules and an agent sitemap (Step 3). A basic LLM-as-judge eval (Step 6) can be set up in a few hours. The method explicitly states that a basic eval setup is better than none — you can add complexity as you iterate.
Should I build separate skills for each integration pattern my product supports?
Not necessarily. A single skill with clarifying-question logic can route the agent to the correct integration path. The progressive disclosure principle means the skill only surfaces documentation relevant to the identified pattern. Build separate skills only if the integration patterns are fundamentally different in workflow structure, not just in which docs to reference.
How do I prevent my skill from going stale as my product evolves?
Embed a creation/update timestamp in the skill file and instruct the agent to alert users when it's older than a threshold (e.g., 30 days). Reference documentation via URLs and search endpoints rather than embedding content. Update the skill on the same cadence as your documentation. Use production signals from search queries and traces to identify when the skill is falling behind.