How to Make Browser Agents Reliable with a Harness

Why do browser agents lie about completing tasks?

Browser-use agents built on non-deterministic models routinely claim they submitted a form, upvoted a post, or completed a checkout when they actually hit a login redirect halfway through. The model generates a plausible success message because that's what it was asked to produce — not because it verified reality. The fix is not a better prompt. It's a harness: everything around the model that grounds it in reality.

The first thing your harness does is remove the lie. Add a verify step — a deterministic function that inspects the agent's trace (the accumulated history of tool calls and events) and returns a definitive pass or fail. For an upvote task, it checks: was there a click on the upvote element, is the page state consistent with success, and were there zero failed-login events? You never ask the model whether it succeeded, because it will tell you yes.

How do I handle login walls without the model touching credentials?

Write a deterministic handler. A `login_handler` fires on every loop iteration before the trace updates, checks the browser's current URL, and if it detects a login page, injects credentials from environment variables and submits the form programmatically — entirely in code. Then it injects a message into the agent's queue: `Harness: I logged in. You are clear to proceed.`

The secret never enters the model's context window. This matters for two reasons: security (a non-deterministic model can leak credentials) and reliability (auth is a step that must be correct, so it belongs in deterministic code, not a probabilistic prompt). Deterministic beats probabilistic for every critical step.

What does the full browser harness look like?

Start with a bare agent loop: while true, get the model response, if STOP return, else push the event to a trace. Attach a tool registry using an existing SDK like OpenAI's tool-calling interface — each tool has a name, description, parameters, and execute function.

Add two guardrails. `max_iterations` of around six kills runaway loops and caps spend. `max_messages` triggers a naive context compressor that keeps the system prompt, user prompt, and last two messages while discarding the middle. Move all logic into `run_harness_attempt`, then wrap it in `run_harness` — a retry loop bounded by `max_attempts`, say three. Your entry point shrinks to about twenty lines.

Finally, add the `login_handler` and the verify step. Run the agent, read the trace when it fails, and add a new handler, guardrail, or verify case. The prompt stays constant the entire time.

Why would I use a cheap model for this?

Because reliability comes from the harness, not the model. A GPT-3.5-level model wrapped in a strong harness completes authenticated browser tasks that an unharnessed frontier model fails. You're doing more with less — exactly the point for engineers who aren't token billionaires with unlimited compute. The model is a swappable black box renter; the harness is the stable anchor you own.

Keep the harness model-agnostic. Don't encode one model's quirks into your deterministic code, so you can swap models freely as prices and capabilities change.

Next step

Pick one failing browser task. Write down exactly how it fails — lying, looping, auth wall — without changing the prompt. Then build the minimal loop, add the two guardrails, write the login_handler for your specific redirect, and add a verify step that checks the trace. Run it, inspect the trace, iterate on the harness. Your cheap model will start completing the task reliably.