How Do DevOps Teams Use Flue to Run AI Agents at Scale?
For DevOps engineers and platform teams · Based on Flue Harness-First AI Agent Framework
// TL;DR
DevOps and platform teams use Flue to deploy thousands of concurrent AI agents at minimal cost using the in-memory just-bash sandbox. Each agent gets a unique instance ID that maps to a Cloudflare Durable Object for isolated persistent state, eliminating shared infrastructure complexity. Use Flue when you need scalable, stateful agent deployments triggered via HTTP or WebSockets — without managing containers, session stores, or tool-loading pipelines manually.
Why Should DevOps Teams Care About Harness-First AI Agents?
Most AI agent frameworks require you to manually wire sandboxes, sessions, memory, and tool loading for every agent. At scale — hundreds or thousands of concurrent agents — this manual wiring becomes a significant infrastructure burden. Flue's harness-first design eliminates this: every agent gets sandboxing, skills, tools, and session management automatically.
For DevOps and platform teams, this means you focus on defining what each agent does (its model, instructions, skills, and tools) rather than how it runs. The framework handles execution infrastructure the same way Claude Code wraps a harness around its model.
How Do You Deploy Thousands of Agents Without Container Overhead?
Flue's default sandbox is just-bash — a TypeScript re-implementation of bash that runs entirely in memory. No container boot time, no per-agent infrastructure cost. This is the key enabler for running thousands of lightweight agents concurrently.
Here's the strategy:
1. Define each agent's scope via skills and tools — keep them focused and minimal.
2. Assign unique instance IDs per job or task.
3. Deploy to Cloudflare Workers with Durable Objects for stateful persistence per instance.
4. Only agents that genuinely require OS-level operations (installing packages, running compiled binaries) opt into a real container sandbox via a URL-specified sandbox endpoint.
The result: most agents run at near-zero marginal cost while the few that need real containers get them on demand.
How Does Cloudflare Durable Object Persistence Work with Flue?
Every running Flue agent has a unique instance ID — any string you assign. When you deploy to the Cloudflare target, this ID maps directly onto its own Durable Object instance. This gives each agent:
- Isolated persistent state that survives Worker restarts
- No shared database or external session store to manage
- Automatic scaling — Cloudflare handles Durable Object placement and lifecycle
On the Node target, the same instance ID enables managing multiple concurrent sessions in memory, but without the built-in persistence that Durable Objects provide.
How Do You Trigger and Monitor Agent Workflows in a CI/CD Pipeline?
Add the root middleware to your Flue project, then build with `flue build` specifying your target and port. The output `server.mjs` inlines all agents and workflows into a single deployable file.
- HTTP POST triggers workflows with a JSON payload
- HTTP GET polls workflow status
- WebSockets stream results in real time for dashboards or monitoring
This integrates cleanly with CI/CD pipelines: trigger a workflow via POST after a deploy, poll for completion, and parse the structured JSON output (response text, token counts, cost, model used) for logging and alerting.
What's the Recommended Project Structure for Platform Teams?
Follow Flue's directory conventions strictly:
- `agents/` — interactive agents requiring human-in-the-loop
- `workflows/` — fully autonomous, repeatable processes
- `skills/` — reusable file-based capabilities with `skill.md` descriptions
- `agent.md` at project root — shared system context across all agents
- `flue.config.ts` at project root — deployment target configuration
Misplacing files is a common pitfall — the CLI resolves paths based on these conventions. Enforce this structure in your repo templates.
Ready to deploy AI agents at scale? Start with `flue init`, choose the Cloudflare target, and build your first workflow in the `workflows/` directory.
// FREQUENTLY ASKED QUESTIONS
Can Flue handle thousands of concurrent agents on Cloudflare?
Yes. The default just-bash sandbox runs in memory with no container boot time or per-agent cost. Each agent gets a unique instance ID mapped to its own Durable Object for isolated persistence. Cloudflare handles scaling and placement automatically, making it practical to run thousands of lightweight agents concurrently.
How do I monitor Flue workflow status in production?
After triggering a workflow via HTTP POST, poll its status with HTTP GET on the same endpoint. The response includes structured JSON with response text, input/output token counts, cost, and model used. For real-time monitoring, connect via WebSocket to stream incremental results to dashboards or alerting systems.
Do I need Docker to run Flue agents?
No. Flue's default sandbox is just-bash — an in-memory TypeScript bash reimplementation requiring no containers. Only opt into a real container sandbox (Daytona, Cloudflare sandbox, or URL-specified) when your agent needs full OS capabilities like package installation or compiled binary execution.