Frequently Asked Questions About Ebbelaar Agentic RAG From Scratch
21 answers covering everything from basics to advanced usage.
// Basics
What does 'agentic' mean in Agentic RAG?
Agentic means the LLM acts autonomously inside a loop, deciding which tools to call, what parameters to pass, and whether it has enough information to answer or needs to search again. Unlike a linear pipeline that runs a fixed sequence, an agentic system lets the model drive its own retrieval process, iterating and self-correcting until the task is satisfied.
What is the Grab tool and how does it work?
Grab is the agent's primary discovery mechanism inside files. It accepts a search pattern and scans all knowledge files, returning matching lines with their file name and line number. In development it uses a case-insensitive compiled regex over splitlines; in production it spawns ripgrep as a subprocess with --line-number --ignore-case --no-heading flags for speed and gitignore-awareness.
What is the difference between List Files and Grab?
List Files gives the agent a map of what knowledge exists by returning the relative paths of all files, while Grab searches inside those files for a specific pattern and returns matching lines. List is for discovery of files; Grab is for discovery of content within files. Read Files then pulls the full content once a relevant file is identified.
What LLM model should I use for the agentic loop?
Use a capable function-calling model like GPT-4o or an equivalent, since the agent depends on strong reasoning to choose tools and search patterns. Weaker models generate poorer parameters and iterate inefficiently. Any OpenAI-compatible provider works. Balance model capability against cost, since each query triggers multiple calls inside the loop, and set a request limit to keep spend predictable.
// How To
How do I set up the List Files tool correctly?
Use Python's pathlib `glob('**/*.md')` to find all markdown files under your notes directory, then return paths with `.relative_to(notes_dir)` rather than absolute paths to minimise token usage. Wrap it in a function with a clear docstring explaining that it lists available knowledge files, since the LLM reads that docstring to decide when to call it.
How do I add citations to my Agentic RAG answers?
Define a typed output model with an `answer: str` field and a `citations: List[Citation]` field, where each Citation contains `file: str`, `quote: str`, and `line_number: int`. Specify this as the agent's required output type. The agent then returns a plain-English answer plus verifiable sources that downstream systems can consume and frontends can render as clickable links.
How do I prevent my agent from looping forever?
Set an agent request limit or max_iterations cap when you wire the tools to the LLM. Without a cap, a confused agent that can't find what it needs will keep issuing tool calls indefinitely, burning tokens and time. A limit like 10 iterations is a reasonable starting point for most knowledge-base queries.
How do I stop large files from overflowing my context window?
Enforce a `read_max_lines` cap (for example 200 lines) in the Read Files tool. Files with thousands of lines can blow up the context window if read in full, so cap the returned content. The agent can always issue additional reads or use Grab to locate specific line ranges rather than pulling an entire large document at once.
// Troubleshooting
My agent keeps crashing when it requests a missing file — what's wrong?
Your tools are likely raising exceptions instead of returning error strings. A raised exception stops the whole agentic loop, so a single missing file crashes the run. Change every tool to return a human-readable error string on failure (like 'File not found') so the LLM can read it and course-correct by trying a different path on its next iteration.
Why is my Agentic RAG surfacing the wrong documents?
The model is probably generating poor search patterns because your tool docstrings lack domain vocabulary. Enable streaming steps to see exactly what terms it searched for, then refine the Grab docstring with hints about useful domain-specific search terms. Docstrings are prompt engineering — the richer the guidance, the better the model's parameter choices and retrieval accuracy.
Why is my Agentic RAG so slow and expensive?
Agentic RAG is inherently slower and costlier than semantic RAG because it makes multiple LLM calls per query inside its loop. If this is a problem, check that you're not letting the agent over-iterate — set a tighter request limit — and confirm you actually need agentic retrieval. If speed and cost are your primary constraints, semantic RAG may be the better fit.
My Grab tool is slow in production — how do I fix it?
Replace the pure-Python regex grep with ripgrep invoked as a subprocess. Python regex reads and scans every file in-process, is slow on large corpora, traverses hidden files, and ignores .gitignore. Ripgrep is a Rust binary that's dramatically faster, skips hidden and git-ignored paths automatically, and matches what Cursor and Claude Code use. Add a startup check to confirm it's installed.
// Comparisons
How does Agentic RAG compare to fine-tuning an LLM on my data?
Agentic RAG retrieves knowledge at query time from a live folder of files, so updates are instant and answers come with citations. Fine-tuning bakes knowledge into model weights, which is expensive to update, offers no source attribution, and risks hallucination. For evolving company knowledge that needs verifiable sources, Agentic RAG is usually the better choice; fine-tuning suits style or behaviour changes, not fact retrieval.
How does Agentic RAG differ from just giving the LLM all my documents in the prompt?
Stuffing all documents into the prompt is limited by context window size, wastes tokens on irrelevant content, and degrades accuracy as content grows. Agentic RAG only pulls the specific lines and files the model needs via Grab and Read, keeping the context focused. It scales to large knowledge bases where the full corpus would never fit in a single prompt.
Should I use LangChain, Pydantic AI, or a hand-rolled loop for Agentic RAG?
Any of the three works — the three-tool pattern is framework-agnostic. Pydantic AI gives you typed output schemas and tool wiring with minimal boilerplate, LangChain offers a broad ecosystem, and a hand-rolled OpenAI function-calling loop gives maximum control and minimal dependencies. Choose based on your team's familiarity and how much abstraction you want between your tools and the model API.
Is ripgrep better than the Python regex Grab for all cases?
For production, yes — ripgrep is faster, respects .gitignore, and skips hidden files. For local development or teaching, the pure-Python regex Grab is fine and requires no external binary. The main reason not to use ripgrep everywhere is deployment complexity: it must be installed in the target environment, so add a startup check before relying on it.
// Advanced
How do I adapt Agentic RAG when my knowledge lives in a database instead of files?
Keep the same three-tool pattern but swap the underlying I/O layer. List Files becomes a query returning available document keys, Grab becomes a full-text or pattern search against the database, and Read Files becomes a fetch by key. The agent's loop and docstring-driven behaviour stay identical — only the storage implementation behind each tool changes.
Can I run Agentic RAG on serverless functions?
Yes. Deploy the agent to a VPS, container app, or serverless function, adjusting only the tool I/O layer if the knowledge source moves. For serverless with ripgrep, ensure the binary is available in the runtime — bundle it or use a layer — and keep the startup check that confirms ripgrep is installed. Watch cold-start latency, which compounds the loop's inherent multi-call latency.
What safety guards are essential before deploying Agentic RAG to production?
Six guards: path containment checks (`is_relative_to`) on all reads, an agent request limit to prevent infinite loops, a `read_max_lines` cap to protect the context window, human-readable error returns instead of raised exceptions, ripgrep replacing the Python grep for speed and gitignore-awareness, and a startup check confirming ripgrep is installed. Add logging around tool calls and total execution time for observability.
How do I optimise retrieval quality once the system works?
Streaming steps are your primary lever. Intercept each tool call to observe what the model searched for, what parameters it chose, and what results returned. When the wrong content surfaces, refine tool docstrings with sharper domain vocabulary so the model generates better search patterns. Iterate query by query, verifying the right files and lines appear before trusting the answers.
Can Agentic RAG generate content, not just answer questions?
Yes. Because the agent retrieves and reads relevant source material, it can draft new content grounded in your knowledge base — for example auto-drafting proposal sections from case study templates. The structured output schema still returns citations back to source documents, giving you an audit trail for the generated text and making it verifiable rather than hallucinated.