How Do DevRel Teams Build Skills That Help AI Agents Use Their Product?
For DevRel and developer experience engineers · Based on Rodrigues Product Skill Architecture Method
// TL;DR
DevRel and developer experience engineers are uniquely positioned to build agent skills because they understand both the product deeply and how developers actually use (and misuse) it. The Rodrigues Product Skill Architecture Method gives DevRel teams a structured process: audit failure modes from community interactions, encode opinionated best-practice workflows directly in skill.md, point agents persistently to live docs, and validate with graded evals. The result is an AI agent that reflects the quality of your developer experience rather than undermining it with stale, incorrect, or unsafe guidance.
Why Should DevRel Teams Own the Agent Skill?
DevRel teams sit at the intersection of product knowledge and developer empathy. You know the common mistakes developers make, the best-practice patterns that work, and the documentation gaps that cause confusion. When AI agents get your product wrong — using deprecated patterns, skipping security steps, or following suboptimal workflows — it directly undermines the developer experience you've worked to build.
The Rodrigues Product Skill Architecture Method turns your DevRel expertise into a testable, versioned skill.md document that closes the context gap between agent training data and your product's current state. Owning this artifact is a natural extension of owning developer experience.
How Do You Audit Agent Failure Modes From Community Interactions?
Your community forums, Discord channels, support tickets, and Stack Overflow tags are a goldmine of agent failure data. Look for patterns:
- Developers asking why an AI-generated code snippet uses a deprecated API
- Security incidents where AI-assisted code missed a required policy flag
- Repeated questions about workflow ordering that suggest agents guide developers through steps in the wrong sequence
- Confusion about which version of a feature to use
Catalog these into a failure mode list. Each entry becomes both a requirement for your skill.md and a test case for your eval suite. This list is the most valuable input to the entire process — do not skip it.
How Do You Encode Best Practices That Agents Will Actually Follow?
The key principle is: if it can be skipped, it will be skipped. Agents are lazy about loading reference files and will rarely load more than one per task. Your best practices have exactly two destinations:
1. skill.md — for anything where missing the guidance produces an incorrect or unsafe outcome
2. Reference files — for genuinely optional supplementary detail
For each best practice, ask: if an agent ignores this and generates code for a developer, will the code be wrong or unsafe? If yes, it goes in skill.md.
Structure your best practices as opinionated workflows — explicit, ordered sequences with rationale:
1. Initialize the project with the recommended template → this ensures correct dependency versions
2. Configure authentication using the platform's auth module → third-party auth packages cause compatibility issues
3. Run the security advisor → catches common RLS misconfigurations
4. Fix all flagged issues before proceeding → prevents security debt from accumulating
The rationale is critical. Agents anchor on reasoning, not just rules.
How Do You Make Agents Use Your Documentation Instead of Training Data?
This is the hardest problem and requires deliberate, persistent instruction. In skill.md, include explicit directives like:
- "Before performing any operation on this platform, fetch the current documentation from [URL]. Do not rely on training data."
- "The API surface has changed since your training cutoff. Always verify endpoint signatures against live documentation."
Repeat these directives at multiple points in the skill — before workflow sections, before security checklists, and in the front matter description. Provide the exact mechanism for accessing docs: URL, docs-over-SSH interface, or search tool endpoint. The repetition counteracts the agent's default behavior of skipping tool calls in favor of cached knowledge.
How Do You Measure Whether Your Skill Improves Developer Experience?
Write eval scenarios that mirror real developer tasks — the same tasks your community asks about. Run each scenario three ways:
- Baseline: no tools, no skill
- MCP-only: tools available but no guidance
- MCP + skill: full stack
Score on graded completeness: correct API usage, security compliance, workflow ordering, and use of current documentation. Track scores across model families.
Share eval results with your product team. The data shows exactly where agents fail and how the skill fixes those failures — this is powerful evidence for prioritizing documentation improvements and API design changes that make your platform more agent-friendly.
What's Your Next Step?
Pull up your community's most common AI-related complaints from the last month. Identify the top three failure patterns. Draft a minimal skill.md addressing just those three, write matching eval scenarios, and run them. You'll have quantitative data on the skill's impact within a day, and a foundation to expand from based on real community needs.
// FREQUENTLY ASKED QUESTIONS
How do DevRel teams find agent failure modes for their product?
Mine your community channels — forums, Discord, support tickets, Stack Overflow — for patterns where AI-generated code fails on your product. Look for deprecated API usage, missed security flags, incorrect workflow sequences, and version confusion. These are direct evidence of the context gap. Catalog each pattern as both a skill.md requirement and an eval test case. Developer-facing teams have the richest data source for this audit.
Should DevRel teams maintain the skill.md or hand it off to engineering?
DevRel teams should own the skill.md because they understand both the product and how developers actually interact with it. The skill is a developer experience artifact, not a purely engineering one. However, coordinate with engineering on security requirements and API changes. Treat the skill like documentation: DevRel writes and maintains it, engineering reviews for technical accuracy, and both teams contribute to the eval suite.
How do I convince my team that building a skill.md is worth the investment?
Run a baseline eval first. Pick three common developer tasks, have an AI agent attempt them without any skill, and document the failures — stale APIs, missed security steps, wrong workflows. Then draft a minimal skill.md, run the same tasks, and show the improvement in graded scores. The before/after comparison is the most compelling argument. Quantified improvement in agent accuracy directly translates to fewer support tickets and better developer experience.