How to Secure Enterprise RAG Agents with a Harness
For Platform engineers building enterprise RAG systems · Based on Tejas Agent Harness Engineering Framework
// TL;DR
Enterprise RAG agents querying sensitive invoices, transcripts, and internal documents can't be trusted to self-police data access or protect credentials. The Tejas Agent Harness Engineering Framework fixes this by making the harness own all secrets and auth tokens — injected deterministically into retrieval tool calls, never placed in the model's context. Guardrails enforce max context size with a compressor, and a deterministic verify step confirms retrieved chunks match the authorised user's data scope before any answer returns. The model stays a replaceable black box; the harness provides enterprise-grade security and reliability as a stable anchor you control.
Why can't the model manage its own credentials and data access?
A model is a non-deterministic black box. If you place auth tokens or user scope rules in its context window, it can leak them, ignore them, or hallucinate access it shouldn't have. For enterprise RAG over sensitive documents — invoices, call transcripts, internal records — that's an unacceptable security posture. The solution is the harness: everything around the model that grounds it in reality and enforces guarantees in deterministic code.
The principle is simple: the harness owns all secrets. Auth tokens are injected deterministically into retrieval tool calls, never handed to the model. The model only ever sees the chunks the harness decides it's allowed to see.
How does the harness enforce data-access scope?
With a deterministic verify step. After retrieval, before any answer is returned, a function inspects the trace and confirms that every retrieved chunk falls within the authorised user's data scope. If a chunk is out of scope, the verify step returns a failure and the answer is never released. This is evidence-based control in code — you never ask the model whether the data was authorised, because a probabilistic system can't be trusted with that decision.
This is the enterprise version of 'deterministic beats probabilistic for critical steps.' Access control is a step that must be correct, so it lives in the harness, not the prompt.
What guardrails does a RAG harness need?
Context management is critical. Long document retrieval quickly blows the context window and degrades performance, so a `max_messages` guardrail must trigger a context compressor. Start naive — keep the system prompt, user prompt, and the last two messages, discard the middle — then improve it incrementally. Don't wait for a perfect compressor to ship; a crude one prevents overflow immediately.
Add a `max_iterations` guardrail to cap retrieval loops and spend. Wrap the logic in `run_harness_attempt` inside a `run_harness` retry loop bounded by `max_attempts`, so transient retrieval failures retry cleanly while permanent scope violations fail fast through the verify step.
How does this let me use a cheaper model?
Reliability and security come from the harness, not the model, so you can run a cheaper or smaller model and still meet enterprise requirements. The model becomes a swappable component — a black box renter — while the harness is the stable anchor providing security, scope enforcement, and context discipline. That's doing more with less, which matters for teams without unlimited frontier-model budget.
Keep the harness model-agnostic so you can swap models for cost or compliance reasons without rewriting your security layer. The deterministic handlers and verify steps don't care which model generated the query plan.
How do I iterate safely in production?
Run the harnessed RAG agent, read the trace on every failure, and add a new verify case or guardrail for each discovered gap — an out-of-scope retrieval, a context overflow, a leaked-looking token path. The prompt stays constant. Each real failure becomes a permanent, auditable piece of your security harness, so your reliability and compliance posture only improve over time.
Next step
Audit where your RAG agent currently touches secrets or makes access decisions. Move every one of those into the harness: inject tokens deterministically into retrieval calls, add a verify step that checks chunk scope before answering, and add a context compressor behind a max_messages guardrail. Run it, inspect the trace, and add verify cases for every gap you find.
// FREQUENTLY ASKED QUESTIONS
Where should auth tokens live in a RAG agent?
In the harness — environment variables or a secure store — injected deterministically into retrieval tool calls. Tokens must never enter the model's context window, because a non-deterministic model can leak or misuse them. The model only receives the chunks the harness has already authorised, keeping all credential handling in deterministic code you control and audit.
How do I prevent the model from returning out-of-scope data?
Add a deterministic verify step that inspects the trace after retrieval and confirms every chunk falls within the authorised user's data scope before any answer is returned. If a chunk is out of scope, the verify step fails and the answer is withheld. You never ask the model to self-enforce access — that decision stays in code.
Do I need a sophisticated context compressor before launching?
No. Start with a naive compressor that keeps the system prompt, user prompt, and last two messages while discarding the middle, triggered by a max_messages guardrail. It prevents context overflow immediately. Ship the harness, confirm it works, then improve the compressor incrementally. Waiting for a perfect compressor is a pitfall that delays a reliable system.