Frequently Asked Questions About Schmid Agent-Ready Engineering Framework
21 answers covering everything from basics to advanced usage.
// Basics
What does 'text is our new state' mean?
It means agents operate on semantic context — text, images, audio, natural-language preferences — rather than Boolean flags and rigid typed schemas. Instead of a churn-flag or a dropdown selection, you pass context-carrying text like 'focus on US market, ignore California.' The model reasons over meaning, so collapsing rich context into typed fields discards information the agent could otherwise use.
What is an eval in AI agent development?
An eval is a probabilistic evaluation method that measures how often an agent succeeds across multiple runs, replacing deterministic unit tests. Evals use qualitative judgment — LLM-as-a-judge or human expert review — plus a reliability threshold like 8 of 10 runs must pass, rather than asserting a single exact output for a single input.
What does 'build to delete' mean for agents?
Build to delete is the 'bitter lesson' principle that agent software is disposable and will be rebuilt — possibly soon — as models improve. Avoid over-investing in brittle scaffolding tied to a specific model's quirks. Design for replaceability, flag tightly coupled components as replacement candidates, and don't fall in love with bespoke workarounds a better model will make obsolete.
What is LLM-as-a-judge and when should I use it?
LLM-as-a-judge is an evaluation technique where a language model scores or qualifies another agent's output, enabling scalable qualitative assessment when exact outputs can't be asserted. Use it for high-volume or qualitative scoring where human review doesn't scale. For high-stakes outputs, supplement or replace it with human expert review to catch judgment errors.
// How To
How do I audit my agent's state management for semantic readiness?
Review every place the agent stores or reads state and ask whether a Boolean flag or rigid structure could instead be natural-language context. Identify user preferences, approval flows, or conditional branches that collapse nuance into a binary, then replace or augment them with context-carrying text so the agent understands semantic meaning rather than just a typed value.
How do I convert a hard-coded workflow into a goal definition?
Locate any orchestration with hard-coded step one, step two, step three logic, then rewrite it as a goal statement plus constraints and let the agent choose the path. Apply the dispatcher metaphor: give the destination and available transport options, not the route. Verify the agent still reaches the outcome even when it takes unexpected intermediate steps — that's acceptable behavior.
How do I add checkpointing to a long-running agent?
Map every point where a tool or API call can fail, then add partial-state preservation after major milestones so a failure at minute 12 doesn't restart from minute 0. For agents running 5+ minutes, this prevents wasted compute and lost context. Combine checkpointing with error-as-input handling so failures feed back to the model rather than triggering full restarts.
How do I set a reliability threshold for my agent?
Define a pass rate your agent must hit before production, such as '9 of 10 runs must pass functional criteria.' Choose the threshold based on stakes — higher for critical flows. Run the agent multiple times against your eval criteria, measure the success rate, and gate deployment on meeting the threshold rather than on any single successful run.
How do I run the observe-adjust loop for agents?
Define instructions, run the agent, observe what it actually does via tracing, adjust prompts or tools, then run again. Schedule deliberate observation sessions instead of assuming correctness after deployment, and track prompt and tool-definition changes like code changes. This iterative loop — not a one-shot deploy — is the core development process for agents.
// Troubleshooting
My agent keeps failing CI even though its output is correct — what's wrong?
Your unit tests are asserting exact deterministic outputs against a non-deterministic system, so the agent appears flaky even when working correctly. Retire exact-output assertions and replace them with evals measuring functional criteria — does it compile, pass functional tests, match requirements? Score with an LLM-as-a-judge or automated harness, set a reliability threshold, and add tracing to see which variance is acceptable.
Why does my agent misroute customer requests and ignore mid-conversation changes?
You're likely using intent classification plus a fixed workflow, which can't adapt when a user changes their mind. Hand over control by replacing the fixed branch with a goal statement like 'resolve the customer's underlying need while preserving the relationship,' and pass the full conversation context instead of a single classification flag so the agent can detect intent shifts.
Why does my research agent waste 15 minutes when one API times out?
It's treating a tool failure as a crash requiring a full restart. Instrument each tool call to catch failures and return them to the model as informational inputs ('search failed with timeout; consider alternative sources'), and add checkpointing after milestones so a late-stage failure doesn't restart from zero. The agent can then reason around the failure and keep moving forward.
Why does the agent misuse my tools even though the code works?
The agent only sees function schemas, doc strings, and tool definitions — it lacks the years of developer context you have. If your doc strings are minimal, the agent can't know what parameters mean or what happens on failure. Rewrite each tool to be fully self-documenting with semantic interfaces, describing every parameter, downstream effect, and error state.
// Comparisons
How does this framework compare to a generic 'just prompt engineer it' approach?
Generic prompt engineering tweaks wording to fix symptoms; the Schmid framework addresses structural causes across state, control, error handling, testing, and tool design. Where prompt tuning treats flakiness as a phrasing problem, this framework recognizes it often stems from rigid workflows, exact-output tests, and under-documented tools — architecture-level issues no amount of prompt polishing resolves.
How is agent engineering different from workflow automation?
Workflow automation defines an explicit deterministic sequence — step one triggers step two — and breaks when reality deviates. Agent engineering defines only the goal and constraints, trusting the LLM to navigate. Automation is a traffic controller dictating every movement; agents require you to be a dispatcher stating the destination and letting the agent choose the path.
How do evals compare to traditional unit tests?
Unit tests assert that one input always produces one exact output, which is correct for deterministic code but wrong for agents. Evals measure how often an agent succeeds across many runs using qualitative judgment and a reliability threshold. Unit tests answer 'is this exactly right?'; evals answer 'how often does this work well enough to trust in production?'
// Advanced
Is this framework only for senior engineers?
No, but it's especially valuable for experienced engineers whose deterministic instincts actively fight agent reliability. The five gaps — rigid workflows, exact-output tests, crash-style error handling, typed state, and under-documented tools — are habits senior engineers hold most strongly. Newer builders benefit too, but the framework specifically targets unlearning deep traditional-engineering reflexes.
How do I balance handing over control with maintaining guardrails?
Hand over the path, not the boundaries. Define the goal and explicit constraints — what markets to focus on, what actions are off-limits — but let the agent choose intermediate steps. Guardrails live in constraints, tool availability, and evals that measure whether outcomes stay acceptable, not in hard-coded step sequences that remove the agent's ability to adapt.
How does 'build to delete' affect long-term architecture decisions?
It pushes you to invest in durable principles over bespoke scaffolding. Couple your architecture to goals, semantic interfaces, and eval criteria that survive model swaps, and avoid deep investment in workarounds tied to a specific model's quirks. Document tightly coupled components as replacement candidates so a model upgrade becomes a swap, not a rewrite of everything.
When is it acceptable for an agent to take unexpected intermediate steps?
It's acceptable and expected whenever the agent still achieves the stated outcome within your constraints. Once you hand over control, the path will vary run to run — that's inherent to non-deterministic systems. Judge success by the outcome via evals, not by whether a specific endpoint was called or a specific route was followed.
How do I decide between LLM-as-a-judge and human expert review?
Use LLM-as-a-judge for scalable qualitative scoring where volume is high and errors are low-stakes or recoverable. Use human expert review for high-stakes outputs where a judgment error is costly. Many teams combine both — LLM-as-a-judge for continuous eval at scale, human review for a sampled subset or for critical production gates.