Frequently Asked Questions About Owain Lewis RAG Retrieval Strategy Selector
22 answers covering everything from basics to advanced usage.
// Basics
What does RAG stand for and what are its three steps?
RAG stands for Retrieval Augmented Generation. Its three steps are: Retrieve (go get relevant information from a data source), Augment (add that information to the LLM prompt), and Generate (produce the answer). The retrieval step is the most important because everything downstream depends on getting the right information — a great LLM can't fix bad retrieval.
What is a chunk in Vector Search?
A chunk is a smaller part that a document is broken into before embedding for Vector Search. Instead of embedding an entire document as one vector, you split it into chunks so each piece is independently embedded, stored, and retrievable. This lets the system return only the most relevant passages rather than whole documents at query time.
What are embeddings and how do they enable semantic search?
Embeddings are numerical vector representations of text produced by an embedding model. Text with similar meaning produces vectors that sit close together in vector space. This lets Vector Search find semantically related content — matching 'comfortable shoes' to 'cushioned midsole, marathon-ready' — by measuring distance between the query vector and stored vectors, where smaller distance means more similar meaning.
What is PGVector and do I need a separate vector database?
PGVector is a PostgreSQL extension that adds vector storage and similarity search to a standard PostgreSQL database. It means you can run Vector Search without deploying a separate dedicated vector database — reducing infrastructure overhead. If you already run PostgreSQL, PGVector lets you handle both structured data and vector retrieval in one place.
// How To
How do I set up Full-Text Search in PostgreSQL?
Use PostgreSQL's built-in TS Vector and TS Query — no extra infrastructure needed. Convert query terms to root forms (stemming) so 'running' matches 'runner' and 'runs,' then use the contains operator to filter records. This handles exact keyword matches like brand names efficiently, but warn users it breaks down when synonyms or semantic meaning matter.
How do I build the full Vector Search pipeline?
Follow five steps: (1) parse documents with a loader like Docling, (2) break documents into chunks, (3) embed each chunk into a vector using an embedding model like OpenAI embeddings, (4) store vectors in a vector database like PGVector, and (5) at query time, embed the user query, measure distance to stored vectors, and return the closest matches.
How do I implement SQL RAG with predefined queries?
Write parameterised SQL queries ahead of time, like 'SELECT * FROM products WHERE category = ? AND price < ? AND rating >= ?'. Use an LLM to extract parameter values from the natural language question (category=running, max_price=100, min_rating=4), inject them into the template, and execute. The query structure never changes, making it deterministic and reliable — ideal for customer-facing products.
How do I set up Agentic RAG correctly?
Give the agent access to all relevant retrieval tools — Document Loading, Full-Text, Vector, and SQL RAG — and crucially provide the database schema so it understands the data structure. The agent inspects the question, decides which tools to invoke, executes them, evaluates results, and self-corrects if wrong. Accept the latency tradeoff and prefer this for internal tooling or complex multi-source questions.
// Troubleshooting
Why is my Vector Search returning products that ignore the price filter?
Vector Search can't handle exact value filters like price, brand, or date — it matches by semantic meaning, not numeric constraints. Asking for 'shoes under $100' may return a $200 shoe that's semantically similar. Fix this by switching to SQL RAG for the filter, or use Hybrid Search combining Vector with structured filtering. Never rely on Vector Search alone for exact-value queries.
Why is my Full-Text Search missing relevant results?
Full-Text Search only matches exact keywords and their stems, so it misses synonyms and paraphrases — 'comfortable' won't match 'cushioned' or 'supportive.' If your queries likely contain varied phrasing, add Vector Search and combine them via Hybrid Search with Reciprocal Rank Fusion. This captures both keyword precision and semantic breadth so relevant results aren't dropped.
My Agentic RAG agent produces useless queries — what went wrong?
You likely failed to give the agent the database schema. Without schema context, the agent can't construct useful SQL queries or select the right tools because it doesn't understand the data structure. Always pass the schema so the agent knows available tables, columns, and relationships. Also check that each retrieval tool is clearly described so the agent can match tools to questions.
My Agentic RAG is too slow for customer-facing use — how do I fix it?
Agentic RAG is slow because the agent makes multiple tool-selection decisions and may retry on failure, which adds latency and non-determinism. If speed is a hard constraint, don't use Agentic RAG — instead pre-select the single right strategy for the question type using the decision tree. Reserve Agentic RAG for internal tooling where latency is acceptable.
// Comparisons
How does Document Loading compare to Vector Search for large document sets?
Naive Document Loading inserts full content into the prompt — accurate but token-heavy and impractical for many documents. For large sets, use Document Loading's index sub-strategy (LLM selects the right document first) or switch to Vector Search, which chunks and retrieves only relevant passages. Choose Document Loading when full content is needed to answer; choose Vector when you need targeted passages from many documents.
How does predefined SQL RAG compare to dynamic LLM-generated queries?
Predefined queries use fixed parameterised templates where only values change — deterministic, reliable, and safe for customer-facing products. Dynamic LLM-generated queries pass the schema to an LLM which writes SQL at runtime — very powerful for ad hoc analysis and internal dashboards but non-deterministic and risky. Use predefined for customer-facing surfaces; reserve dynamic queries for internal analytics and reporting.
How does this decision-tree approach compare to just defaulting to Vector Search?
Defaulting to Vector Search treats every retrieval problem the same and fails on exact filters, purely keyword queries, and full-document needs. The decision-tree approach diagnoses data shape and question type first, then matches the right strategy — Document Loading, Full-Text, Vector, Hybrid, SQL, or Agentic. This avoids common retrieval failures and often reveals that a simpler, cheaper strategy solves the problem better.
How does Hybrid Search compare to running Vector Search alone?
Hybrid Search runs Full-Text and Vector in parallel and merges results via Reciprocal Rank Fusion, capturing both exact keyword precision and semantic meaning. Vector alone catches meaning but misses exact keyword matches like brand names, and can't rank keyword relevance well. For queries mixing keywords and meaning — 'Nike running shoes comfortable for distance' — Hybrid outperforms Vector alone and is the safer production default.
// Advanced
When should I choose Document Loading over building a Vector Search pipeline?
Choose Document Loading when you have few documents and the full content is needed to answer correctly — policy documents, runbooks, checklists, or recipes where partial retrieval breaks the answer. It's simpler, more accurate, and avoids embedding infrastructure. Only build a Vector Search pipeline when documents are numerous and you need to retrieve targeted passages rather than entire documents.
How do I handle a compound question spanning documents and a database?
Use Agentic RAG. For a question like 'I want running shoes under $150 — and what's your return policy?', give the agent both SQL RAG (product filter by price and category) and Document Loading (returns policy). The agent invokes the SQL tool for the shoe search, the document loader for the policy, then combines both results into one coherent answer.
What is Reciprocal Rank Fusion and how does it merge results?
Reciprocal Rank Fusion (RRF) is an algorithm used in Hybrid Search to merge ranked result lists from Full-Text Search and Vector Search into a single unified ranking. It scores each result based on its position across both lists, so items ranked highly by either method rise to the top. This balances keyword precision and semantic relevance without manually tuning weights.
Should I ever combine SQL RAG with Vector Search?
Yes — when a query mixes exact filters with semantic matching, combine them. For example, filter products in SQL by price and category, then rank the filtered set semantically with Vector Search. Alternatively, expose both as tools in an Agentic RAG setup and let the agent orchestrate. This handles queries where structured constraints and meaning both matter.
How do I validate my chosen strategy before shipping?
Check known failure modes: for Document Loading, confirm you can identify which document to load or use the index sub-strategy; for Full-Text, check if synonyms are likely and add Vector if so; for Vector, check for exact filters and add SQL if present; for dynamic SQL RAG, prefer predefined queries if customer-facing; for Agentic RAG, pre-select a single strategy if latency is a hard constraint.
Can I start simple and add complexity later?
Yes — always consider whether the simplest approach solves the problem before adding complexity. Document Loading and Full-Text Search are underrated and often sufficient. Start with the strategy the decision tree recommends, validate it against failure modes, and only escalate to Hybrid or Agentic RAG when your queries genuinely require semantic breadth or multi-source retrieval. Complexity adds latency, cost, and non-determinism.