Ebbelaar Agentic RAG From Scratch

Last updated: 18 July 2026

Build a production-ready Agentic RAG system using three core file-system tools — List, Grab, Read — that lets an LLM self-correct in a search loop to retrieve private or company knowledge far more accurately than semantic RAG.

// TL;DR

Ebbelaar Agentic RAG From Scratch is a method for building a production-ready retrieval system where an LLM sits inside a search loop with three file-system tools — List Files, Grab, and Read Files — so it can self-correct and re-search until it finds the right answer in your private markdown knowledge base. Use it when you need accurate retrieval over company, domain, or private knowledge and can afford multiple LLM calls per query. Skip it when raw speed or minimal cost are your top priorities — start with semantic RAG in those cases. It delivers verifiable, cited answers ready for downstream automation.

Framework

// When should you use Ebbelaar Agentic RAG From Scratch?

Use this skill when you need to make private, company, or domain-specific knowledge available to an LLM-powered automation and you have the latency budget and cost tolerance for multiple LLM calls. Do NOT use it when raw speed or minimal cost are the primary constraints — start with semantic RAG in those cases.

// What do you need before building an Agentic RAG system?

Knowledge Sourcerequired
A folder of markdown (.md) files containing the domain knowledge the agent should search over (e.g. engineering wiki, runbooks, decision logs, product docs).
LLM Provider + API Keyrequired
An OpenAI-compatible API key or any LLM provider you want to power the agentic loop.
Agent Framework Choicerequired
The framework used to wire tools to the LLM loop — e.g. Pydantic AI, LangChain, or a hand-rolled loop using the model API directly.
User Queryrequired
The natural-language question or task the agent must answer using the knowledge source.
Output Schema (optional)
A structured data model defining how the answer should be returned, e.g. answer + citations with file, quote, and line number.
Production Environment Target (optional)
Deployment context — local, VPS, container app, or serverless function — so tool dependencies (e.g. ripgrep installation) can be adjusted.

// What are the core principles behind Agentic RAG?

Agentic Loop vs. Linear RAG

Semantic RAG calls the LLM only once after retrieval — intelligence is used a single time. Agentic RAG places the LLM inside a feedback loop with List, Grab, and Read tools so it can self-correct, re-search, and iterate until it finds the right information. More LLM calls = higher accuracy.

Three Core Primitives

Every agentic RAG system — and every popular agent harness (Cursor, Claude Code, etc.) — is built around the same three tools: List Files (discover what exists), Grab (search for patterns inside files), and Read Files (retrieve full content). Master these three and you can build any knowledge agent.

Tool Docstrings as Agent Instructions

Everything written in a tool's definition and docstring is information the LLM uses to decide when to call the tool, what parameters to pass, and how to interpret results. Treat docstrings as first-class prompt engineering — add domain hints and parameter guidance directly inside them.

Return Errors, Don't Raise Them

In production, tool errors should return human-readable error messages rather than raise exceptions. Raising stops the agent process entirely; returning allows the LLM to intercept the error message and course-correct on its next loop iteration.

Contain the Agent with Path Safety

Always validate that any file path the agent attempts to read is relative to the designated notes/knowledge directory using an `is_relative_to` check. This prevents the agent from escaping its sandbox and reading arbitrary files on the host system.

Structured Output with Citations

Wrap the agent's final answer in a typed output schema: a plain-English answer field plus a citations list (file name, exact quote, line number). This makes the result machine-consumable for downstream systems and enables verifiable, clickable source attribution in frontends.

Ripgrep for Production Grab

Replace a pure-Python regex grep with ripgrep (rg) — a Rust-based subprocess — for production deployments. Ripgrep automatically skips hidden files and git-ignored paths, is significantly faster, and matches what real agent harnesses like Cursor and Claude Code use internally.

Debug with Streaming Steps

When optimising or debugging the agentic loop, intercept tool calls to observe: what the model searched for, what parameters it chose, and what results came back. Inspect this to verify the right documents surface for a given query — this is the primary optimisation lever.

// How do you build an Agentic RAG system step by step?

1
Prepare your knowledge source as markdown files in a single directory
Markdown is the easiest format for this agentic loop. Create or export your domain knowledge (runbooks, wikis, decision logs, product docs) as .md files in one folder. Name the folder clearly (e.g. /notes or /knowledge). For non-local storage (Postgres, S3), the same three-tool pattern applies — you just swap the underlying I/O functions.
2
Build the List Files tool
Use Python's pathlib `glob('**/*.md')` to find all markdown files under the notes directory. Return paths using `.relative_to(notes_dir)` — NOT full absolute paths — to keep token usage minimal. Wrap in a function with a clear docstring explaining it lists available knowledge files.
3
Build the Grab tool
Implement a pattern-search tool that accepts a search string and searches across all files for matching lines. For simple/dev use: compile a case-insensitive regex with `re.compile(pattern, re.IGNORECASE)`, read each file, `splitlines()`, enumerate from 1, and collect (file_name, line_number, line_text) tuples for every match. For production: spawn ripgrep as a Python subprocess with flags `--line-number --ignore-case --no-heading`. The docstring must explain what patterns are useful and hint at domain-specific search terms to guide LLM parameter selection.
4
Build the Read Files tool
Accept a relative file path, resolve it to an absolute path, then verify it passes `target.is_relative_to(notes_dir)` before reading. In production, add a `read_max_lines` cap (e.g. 200 lines) to avoid blowing up the context window with very large files. Return the file content as a string; return a human-readable error string (do not raise) if the path check fails or the file is not found.
5
Wire the three tools to an LLM inside an agentic loop
Register List Files, Grab, and Read Files as callable tools on your chosen agent framework (Pydantic AI, LangChain, custom OpenAI function-calling loop, etc.). Set the model (e.g. GPT-4o or equivalent). Set a `max_iterations` / agent request limit to prevent infinite loops. Pass the user's query. The LLM will autonomously decide which tools to call, what parameters to use, and will iterate until it has sufficient information to answer.
6
Add a structured output schema with citations
Define a typed output model with at minimum: `answer: str` (plain English response) and `citations: List[Citation]` where Citation contains `file: str`, `quote: str`, `line_number: int`. Specify this as the agent's required output type. This makes results usable by downstream systems and enables frontend citation components.
7
Enable streaming steps / debug mode for inspection and optimisation
Intercept tool calls during execution to log: (a) which tool was called, (b) what parameters the LLM chose to pass, (c) what results were returned. For each query, verify the right documents and lines are surfacing. If they are not, refine tool docstrings with better domain vocabulary or adjust the search patterns the model is likely to generate.
8
Harden for production with safety guards and ripgrep
Add: (1) path containment checks on all file reads, (2) agent request limits, (3) read_max_lines cap, (4) human-readable error returns (never raises) for all edge cases, (5) ripgrep subprocess replacing the Python regex Grab for speed and .gitignore-awareness, (6) a startup check confirming ripgrep is installed in the environment. Add logging around tool calls and total execution time. Deploy as-is to VPS, container app, or serverless — adjust only the tool I/O layer if the knowledge source moves to a database.

// What are real-world examples of Agentic RAG in action?

A SaaS company has 50 internal runbooks and architectural decision records as markdown files and wants engineers to query them in natural language via a Slack bot.

Point the notes directory at the runbooks folder. Build List, Grab, and Read tools against it. Wire to GPT-4o with a 10-iteration limit. The structured output schema returns the answer plus citations (file name + line number) so the Slack bot can link directly to the relevant runbook section. Use streaming steps logging during QA to verify the model is correctly searching for terms like 'deployment window' or 'database migration' and surfacing the right files.

A consulting firm wants to query a library of client proposal templates and case study documents to auto-draft new proposal sections.

Export all templates and case studies as .md files into a /knowledge folder. The Grab tool lets the agent search for industry-specific terminology or service names across all documents. The Read tool pulls the relevant sections in full. The structured output includes the drafted section plus citations back to the source documents, giving consultants an audit trail. In production, ripgrep handles the large file corpus efficiently without slowing the loop.

// What mistakes should you avoid when building Agentic RAG?

Using full absolute file paths in List Files responses — this wastes tokens unnecessarily; always return relative paths using `.relative_to(notes_dir)`.
Raising exceptions instead of returning human-readable error strings from tools — a raised exception stops the agentic loop entirely, preventing self-correction.
Omitting the path containment (`is_relative_to`) check in the Read Files tool — without it the agent can be prompted or hallucinate its way into reading arbitrary system files.
Not setting an agent request limit / max iterations — without a cap, a confused agent can loop indefinitely, burning tokens and time.
Reading entire large files without a line cap — files with thousands of lines can blow up the context window; enforce `read_max_lines` in production.
Writing vague or empty tool docstrings — the LLM uses docstrings to decide what to search for and what parameters to pass; poor docstrings produce poor search behaviour.
Using the Python regex Grab tool in production — it is slower, does not respect `.gitignore`, and traverses hidden files; replace with ripgrep via subprocess for any production deployment.
Choosing Agentic RAG when latency or cost is the primary constraint — multiple LLM calls per query make it slower and more expensive than semantic RAG; only use it when accuracy improvement justifies the trade-off.
Not inspecting streaming steps during development — skipping debug mode means you cannot see what the model is actually searching for, making it impossible to optimise tool docstrings or diagnose retrieval failures.

// What are the key terms in Agentic RAG explained?

Agentic RAG: A retrieval-augmented generation architecture where the LLM is placed inside a feedback loop with search and read tools, allowing it to self-correct and re-query across multiple iterations rather than making a single retrieval pass.
Semantic RAG: The classical, linear RAG approach: retrieve relevant chunks via vector/semantic similarity in one pass, then call the LLM once with all retrieved context. Faster and cheaper than Agentic RAG but cannot self-correct.
Agentic Loop: The iterative execution cycle in which the LLM calls tools (List, Grab, Read), receives results, decides if it has enough information, and either answers or issues another round of tool calls until the task is satisfied.
List Files (tool): The first core primitive: discovers which markdown files exist in the knowledge directory and returns their relative paths, giving the agent a map of available knowledge.
Grab (tool): The second core primitive: searches across all files for a pattern (keyword, phrase, regex) and returns matching lines with their file name and line number. The agent's primary discovery mechanism for locating relevant content within files.
Read Files (tool): The third core primitive: retrieves the full (or capped) text content of a specific file after the agent has identified it as relevant via List or Grab.
Ripgrep (rg): A Rust-based command-line search tool invoked as a Python subprocess in production. Faster than Python regex grep, automatically skips hidden files and git-ignored paths, and is the same tool used by Cursor, Claude Code, and other modern agent harnesses.
Structured Output / Citations: A typed response schema requiring the agent to return both a plain-English answer and a list of citation objects (file, quote, line number), enabling downstream system integration and verifiable source attribution.
Streaming Steps: A debug/inspection mode that intercepts each tool call during the agentic loop to expose what tool was called, what parameters the LLM chose, and what results were returned — the primary instrument for optimising retrieval quality.
Return Errors, Don't Raise: A production best practice where tool functions return human-readable error strings on failure rather than raising Python exceptions, allowing the LLM to read the error and course-correct without crashing the agent process.
Path Containment Check: A safety guard using `path.is_relative_to(notes_dir)` that verifies any file the agent attempts to read is inside the designated knowledge directory, preventing sandbox escape.
Agent Request Limit: A configurable maximum number of tool-call iterations the agent is allowed to make per query, preventing infinite loops when the agent cannot find what it needs.
Read Max Lines: A configurable cap on how many lines the Read Files tool returns from any single file, preventing large documents from overflowing the LLM's context window.

// FREQUENTLY ASKED QUESTIONS

What is Agentic RAG?

Agentic RAG is a retrieval-augmented generation architecture where the LLM is placed inside a feedback loop with search and read tools, letting it self-correct and re-query across multiple iterations instead of a single retrieval pass. This makes it far more accurate than classical semantic RAG for private or domain-specific knowledge, at the cost of more LLM calls, higher latency, and higher spend per query.

What are the three core tools in an Agentic RAG system?

Every Agentic RAG system uses three primitives: List Files (discover which knowledge files exist), Grab (search for a pattern inside files and return matching lines with line numbers), and Read Files (retrieve the full or capped content of a specific file). These same three tools power modern agent harnesses like Cursor and Claude Code — master them and you can build any knowledge agent.

How do I build an Agentic RAG system from scratch in Python?

Prepare your knowledge as markdown files in one folder, then build three tools: List Files using pathlib glob, Grab using regex or ripgrep, and Read Files with a path-containment check. Wire all three to an LLM inside an agentic loop with a max-iterations cap, add a structured output schema with citations, and harden with safety guards and ripgrep before deploying.

How do I stop an Agentic RAG agent from reading files outside its knowledge folder?

Add a path containment check to the Read Files tool using `target.is_relative_to(notes_dir)` before reading any file. Resolve the relative path to absolute, verify it falls inside the designated knowledge directory, and return a human-readable error string if it doesn't. This prevents the agent from being prompted or hallucinating its way into reading arbitrary system files.

How does Agentic RAG compare to semantic RAG?

Semantic RAG retrieves chunks via vector similarity in one pass and calls the LLM once — it's fast and cheap but can't self-correct. Agentic RAG puts the LLM inside a loop with List, Grab, and Read tools so it iterates until it finds the right information, delivering higher accuracy at the cost of more LLM calls, latency, and spend. Choose based on whether accuracy or speed matters most.

When should I use Agentic RAG instead of semantic RAG?

Use Agentic RAG when you need accurate retrieval over private, company, or domain-specific knowledge and you have the latency budget and cost tolerance for multiple LLM calls per query. Don't use it when raw speed or minimal cost are the primary constraints — start with semantic RAG in those cases and only upgrade if accuracy proves insufficient.

What results can I expect from Agentic RAG?

You get significantly more accurate retrieval on private knowledge than semantic RAG, because the LLM can re-search and self-correct until it finds the right content. With a structured output schema you also get verifiable answers plus citations (file, exact quote, line number) that downstream systems can consume and frontends can render as clickable sources. The trade-off is higher latency and cost per query.

Why should tool errors return messages instead of raising exceptions in Agentic RAG?

Returning human-readable error strings lets the LLM read the error and course-correct on its next loop iteration, while raising an exception stops the agent process entirely and prevents self-correction. For example, if the agent requests a nonexistent file, a returned 'file not found' message lets it try a different path — a raised exception just crashes the run.

Do I need ripgrep for Agentic RAG?

You don't need ripgrep for development, where a pure-Python regex grep works fine, but you should use it in production. Ripgrep is a Rust-based subprocess that's significantly faster, automatically skips hidden files and git-ignored paths, and is the same tool Cursor and Claude Code use internally. Add a startup check confirming it's installed in your deployment environment.

What knowledge format works best with Agentic RAG?

Markdown (.md) files in a single, clearly named directory (like /notes or /knowledge) are the easiest format for the agentic loop. Export your runbooks, wikis, decision logs, or product docs as markdown. If your knowledge lives in Postgres or S3 instead, the same three-tool pattern applies — you simply swap the underlying I/O functions in List, Grab, and Read.

How do I debug why my Agentic RAG agent isn't finding the right documents?

Enable streaming steps to intercept each tool call and log which tool was called, what parameters the LLM chose, and what results came back. For each query, verify the right files and lines surface. If they don't, refine your tool docstrings with better domain vocabulary or adjust the search patterns the model is likely to generate — this is your primary optimisation lever.

Why are tool docstrings so important in Agentic RAG?

The LLM uses everything in a tool's definition and docstring to decide when to call it, what parameters to pass, and how to interpret results. Treat docstrings as first-class prompt engineering — add domain hints and parameter guidance directly inside them. Vague or empty docstrings produce poor search behaviour and bad retrieval, so specify useful patterns and domain-specific search terms.

// GET THIS SKILL — FREE