How to Write AI-Facing Docs Agents Actually Follow

Why do AI agents keep misusing your SDK?

Because they do not know your product's implicit contracts — not because they cannot code. Models are fluent in your language and framework; what they lack is knowledge of the specific landmines unique to your SDK: the initialization order that must not change, the method that silently fails without a config flag, the edge case that breaks across versions. Dumping your entire documentation into context does not fix this. It buries the critical gotchas under thousands of lines the model already knows, and more context can actively degrade performance.

What should you put in an AI-facing skill?

Apply guide, don't prescribe: surface only the gotchas. A gotchas file is a targeted, minimal list of the implicit contracts and failure points agents reliably hit in your product. Aim for under 600 lines. Do not convert your reference docs wholesale — that is the single most common mistake teams make when building AI-facing skills. The goal is to expose what the model cannot know, not to re-teach programming fundamentals it already has.

How do you find out which gotchas actually matter?

Apply measure, don't assume with an eval suite. Build a set of tasks that exercise your SDK, run agents against them, and record the pass rate. The failures tell you exactly which behaviors the model gets wrong — those become your gotchas. This is empirical, not guesswork: you are not writing down everything you think is tricky, you are writing down what demonstrably breaks. Without evals you cannot tell an improvement from noise, and you will ship skill content that silently lowers pass rate.

How do you know your gotchas file actually helps?

Re-run evals after writing it. Confirm the gotchas file raises pass rate — and crucially, that it does not introduce new failures. More content is not better; 553 lines of targeted gotchas can outperform 10,000 lines of comprehensive docs. If any line of your skill reduces pass rate, delete it. Treat your AI-facing docs as a living artifact optimized against a score, not a static reference ported from your human docs.

How do these gotchas plug into a real agent pipeline?

In a harness, relevant gotcha memory files are loaded at the start of a task — and only the relevant ones, because irrelevant context degrades performance. A retrospective agent reads each run's transcript and appends newly discovered landmines to the appropriate per-framework memory file, so your gotchas improve automatically as real agents use your SDK in the wild. Over time your product accumulates earned, verified guidance rather than aspirational documentation.

Next step

Stand up a small eval suite of realistic SDK tasks, run current agents against it to capture your baseline pass rate, and list every failure. Convert those failures — and only those — into a sub-600-line gotchas file, then re-run evals to prove the lift. Publish it as your official AI-facing skill and wire it into a retrospective loop so it keeps improving.