Frequently Asked Questions About Bri's Production Prompt Engineering Method
21 answers covering everything from basics to advanced usage.
// Basics
What does 'production' mean in production prompt engineering?
Production means the agent is running live against real users and real data, where inconsistent outputs cause actual harm — wrong escalations, bad answers, or refusals. Production prompt engineering prioritizes repeatable, predictable behavior over clever one-off prompting, because you can't manually review every response. Evals and declared output shapes exist to guarantee consistency at scale.
What is Decision Criteria and how is it different from a prohibition?
Decision Criteria is a calm, positive instruction that tells the agent when to take one action versus another, and why. A prohibition ('never escalate unless necessary') signals urgency and distorts judgment, while decision criteria ('escalate when the issue is a billing dispute; keep resolving when it's a product question') let the model evaluate neutrally without strong-word signals overriding its decision.
What is the harness in an AI agent system?
The harness is the code layer that wraps and manages the agent. It injects dynamic runtime data into the prompt as variables, calls tools, and passes computed results back to the model. Anything that shouldn't live in the static prompt — live data, deterministic calculations, lookups — belongs in the harness. The prompt stays stable; the harness handles what changes.
What inputs do I need before starting an audit?
At minimum, the full current prompt including system instructions and role definitions, plus a plain-English description of the agent's purpose and what a correct output looks like. Optionally, a description of the observed failure modes and a log of patches added over time. The patch history and failure descriptions make the audit faster and more targeted, but the prompt and purpose are mandatory.
// How To
How do I set up eval tests before iterating on a prompt?
Create a test suite that makes real API calls with your current prompt against representative inputs, then record what passes and what fails. This becomes your baseline. Define checks for the things that matter — correct decision made, correct response length, correct format. Run each test multiple times because model calls vary, and compare against baseline after every change.
How do I rewrite a prohibition as decision criteria step by step?
Find the prohibition, identify the situation it was trying to prevent, then state the correct behavior for both that situation and its opposite. For example, replace 'never resolve billing disputes yourself' with 'escalate when the issue is a billing dispute; continue resolving when it's a general product question.' Remove all urgency language — 'never,' 'critical,' 'always' — so the model evaluates the criteria calmly.
How do I structure a prompt with HTML tags?
Separate distinct sections with HTML-style tags like <role>, <instructions>, and <reply>. Put the agent's identity in <role>, its behavior rules in <instructions>, and its exact output format in <reply>. This gives the model clear boundaries between what it is, what it should do, and how it should respond, producing more consistent parsing and output than an unstructured block of text.
How do I decide what to cut from a bloated prompt?
Cut anything the model already knows by default, all filler and redundant explanations, and any instruction you can't tie to a specific failure. For patches, if you can't name the exact failure a rule prevents, cut it first. Resolve conflicting instruction pairs by keeping one. Shorter and cleaner consistently outperforms longer and more detailed.
How do I know when the cleanup is done and the prompt is good?
Re-run your eval suite multiple times, since model calls vary, and look for a short, clear, definitive answer that matches your declared output shape consistently across runs. If outputs are stable and correct against your baseline, you're done. If it still fails, return to auditing prohibitions and structure — do not add new instructions until cleanup possibilities are exhausted.
// Troubleshooting
My agent refuses to do things it's clearly capable of. How do I fix it?
This is over-caution, caused by defensive instructions the agent treats as top priority rather than guardrails. Audit the prompt for prohibitive language and 'critical' warnings, then rewrite each as neutral decision criteria. Remove urgency signaling entirely. Newer models are especially prone to this because they follow instructions more literally and abstain rather than guess.
My prompt worked fine, then broke after a model upgrade. Why?
Newer models follow instructions more literally and will abstain rather than guess, breaking prompt patterns that older models glossed over. Prohibitions that older models loosely interpreted now dominate behavior. Run a full patch audit and rewrite all prohibitions as decision criteria before upgrading, and re-run your eval suite against the new model to catch literal-interpretation failures.
My agent's answers vary wildly in length and style. What's wrong?
You likely haven't declared an exact output shape. Without a declared format, length and style drift run to run. Add a <reply> tag specifying format, length, and structure — for example, 'respond in one to two sentences with a direct, factual answer.' Models follow declared output shapes faithfully, so this alone usually stabilizes response consistency.
I added instructions to fix a bug but now the agent is more unpredictable. Why?
Patches accumulate and eventually contradict each other; the model tries to enforce all of them and no one can predict which wins. Instead of adding, apply Cleanup First — run a patch audit, resolve conflicting pairs, remove stale instructions whose root cause is already fixed. Don't add new instructions until cleanup possibilities are exhausted.
// Comparisons
How does this method compare to generic prompt engineering tips?
Generic tips optimize for a single impressive output; Bri's method optimizes for repeatable behavior in production across many runs. It's opinionated: no prohibitions, declared output shapes, dynamic data in the harness, deterministic work offloaded to tools, and mandatory evals. Generic advice rarely addresses patch accumulation, one-sided trade-offs, or model-upgrade fragility — the failure modes that actually break production agents.
How does this differ from few-shot prompting or chain-of-thought?
Few-shot and chain-of-thought are techniques for improving reasoning quality on individual tasks; Bri's method is a discipline for making agent behavior predictable in production. You can use few-shot examples inside this framework, but the method's focus is structural — decision criteria over prohibitions, declared outputs, harness-injected data, and eval-driven iteration — not reasoning tricks.
Is this method better than just using a bigger or newer model?
Upgrading models doesn't fix bad prompt structure — it often makes it worse, because newer models follow instructions more literally and expose prohibitions and conflicting patches that older models glossed over. Bri's method is what makes a model upgrade safe: audit patches, rewrite prohibitions, and re-run evals so the more literal model behaves predictably.
// Advanced
What is balanced trade-off framing and why does it matter?
Balanced trade-off framing means stating the cost of both choices in any judgment call, not just one. If the prompt only mentions the cost of one option, the agent consistently drifts toward avoiding that cost — becoming too aggressive or too conservative. Presenting both costs lets the model weigh them and make context-appropriate decisions instead of reflexively erroring in one direction.
How do I handle deterministic computation that the agent currently does?
Remove that responsibility from the prompt entirely. Implement it as a tool call the agent invokes, or compute it directly in the harness and inject the result as a variable. LLMs are unreliable at deterministic work like math, lookups, and ID generation, producing different answers across runs. Offloading it guarantees a single correct value every time.
What's the difference between a stale instruction and a live patch?
A live patch fixes a bug whose root cause still exists and you can name the failure it prevents. A stale instruction is a patch whose original root cause — like a data pipeline bug — has been fixed, but the instruction was never removed. The model still enforces stale instructions faithfully, distorting behavior for no reason. Patch audits catch and remove them.
How often should I run a patch audit?
Run a patch audit whenever a prompt has accumulated fixes over time, whenever behavior becomes unpredictable, and always before a model upgrade. As a cadence, review any production prompt periodically to catch stale instructions and conflicting pairs before they compound. Treat every new patch as debt that requires documentation — what failure it prevents — so future audits are fast.
Can I use decision criteria and still keep safety guardrails?
Yes — guardrails become decision criteria instead of prohibitions. Rather than 'never share account data,' write 'share account data when the request is authenticated and matches the account owner; decline and explain when it doesn't.' This keeps the safety behavior intact while removing the urgency signaling that causes over-caution. The guardrail is stronger because it tells the model exactly when each behavior applies.