How to Build Agentic RAG From Scratch in Pure Python

Why build Agentic RAG from scratch instead of using a library?

Building it yourself teaches you the pattern behind every modern agent harness. Cursor, Claude Code, and other tools are all built around the same three primitives — List Files, Grab, and Read Files. Once you understand how these wire to an LLM loop, you can build any knowledge agent and debug any framework's abstraction. A hand-rolled OpenAI function-calling loop keeps dependencies minimal and gives you full control over iteration limits, error handling, and output schemas. You're not locked into LangChain or Pydantic AI conventions — though either works fine if you prefer their scaffolding.

What are the three tools you need to implement?

List Files uses pathlib `glob('*/.md')` and returns relative paths via `.relative_to(notes_dir)` to keep token usage low. Grab searches all files for a pattern and returns (file, line number, line text) tuples — a case-insensitive compiled regex for development, or ripgrep via subprocess for production. Read Files takes a relative path, resolves it, verifies `is_relative_to(notes_dir)`, and returns capped content. Each tool's docstring is prompt engineering: it tells the LLM when to call the tool and what parameters to pass, so write them with domain hints.

How do you wire the tools into an agentic loop?

Register List, Grab, and Read as callable tools on your chosen framework or in a raw function-calling loop. Set the model — GPT-4o or equivalent — and a `max_iterations` cap to prevent infinite loops. Pass the user query and let the LLM autonomously decide which tools to call, what parameters to use, and when it has enough to answer. Then define a typed output schema with an `answer` field and a `citations` list (file, quote, line number) so the result is machine-consumable and every claim is verifiable.

How do you debug and optimise the agent?

Streaming steps are your primary instrument. Intercept each tool call to expose what tool was called, what parameters the model chose, and what results returned. For every test query, verify the right documents and lines surface. When they don't, the fix is almost always sharper tool docstrings — better domain vocabulary steers the model toward better search patterns. Skipping debug mode means flying blind: you can't see what the model searched for, so you can't diagnose retrieval failures or improve them.

What separates a demo from a production Agentic RAG?

Six guards. Path containment checks on every read confine the agent to its sandbox. An agent request limit stops runaway loops. A `read_max_lines` cap protects the context window from huge files. Human-readable error returns — never raised exceptions — let the LLM course-correct instead of crashing the process. Ripgrep replaces the Python regex Grab for speed and .gitignore-awareness. A startup check confirms ripgrep is installed. Add logging around tool calls and total execution time, then deploy to a VPS, container app, or serverless function — swapping only the tool I/O layer if your knowledge moves to a database.

When should you reach for Agentic RAG versus semantic RAG?

Reach for Agentic RAG when accuracy over private knowledge matters and you can tolerate multiple LLM calls' worth of latency and cost. Its self-correcting loop beats semantic RAG's single-pass retrieval on hard queries. But if raw speed or minimal spend is your primary constraint, start with semantic RAG and only upgrade when accuracy proves insufficient. The two aren't mutually exclusive — many production systems route easy queries to semantic RAG and hard ones to the agentic loop.

Next step: Create a /notes folder of markdown, implement List, Grab, and Read with the six production guards, and wire them to a function-calling loop with a citation schema — then watch the streaming steps to tune your docstrings.