How to Pick the Right RAG Strategy for a Support Bot
For SaaS founders building support chatbots · Based on Owain Lewis RAG Retrieval Strategy Selector
// TL;DR
If you're a SaaS founder building an AI support chatbot, this framework helps you choose the correct RAG strategy so your bot answers policy, product, and account questions accurately. For refund and shipping policies stored in PDFs, use Document Loading. For product searches with filters, use SQL RAG with predefined queries. For compound questions like 'find shoes under $150 and tell me your return policy,' use Agentic RAG. Diagnosing data shape and question type first prevents the wrong-answer failures that erode customer trust and inflate support tickets.
What RAG strategy should my support chatbot use?
It depends on where your answers live and what customers ask. Most SaaS support content splits into two buckets: policy documents (refunds, shipping, terms, onboarding runbooks) and structured data (subscriptions, orders, account details in your database). The RAG Retrieval Strategy Selector diagnoses your data shape and question type, then routes you to one of six production-tested strategies instead of forcing everything through Vector Search — the default that quietly fails on exact filters and full-document lookups.
How do I handle policy questions like refunds and shipping?
Use Document Loading. If your refund and shipping policies live in two PDFs, load the entire relevant document into the prompt — the full policy text is needed to answer correctly, and partial retrieval breaks the answer. Add the index sub-strategy: maintain a small index of filenames plus descriptions so an LLM first selects the right document (refund versus shipping) before loading it. This avoids burning tokens on irrelevant documents while keeping answers accurate.
Document Loading gets dismissed as naive, but for policy documents, checklists, and runbooks it's surprisingly reliable and underrated. Don't over-engineer a Vector Search pipeline when loading two PDFs solves the problem cleanly.
How do I let customers search products or account data?
Use SQL RAG with predefined queries. When a customer asks 'Show me running shoes under $100 with at least four stars,' the data lives in a structured database and the query involves exact filters. Write a parameterised query ahead of time — `SELECT * FROM products WHERE category = ? AND price < ? AND rating >= ?` — extract the parameters from the natural language question with an LLM, inject them, and execute.
This is deterministic and reliable, which is exactly what you want on a customer-facing surface. Avoid dynamic LLM-generated SQL here — it's non-deterministic and risky. Reserve dynamic queries for your internal analytics team, not your customer chatbot.
What about compound questions that mix policy and products?
Use Agentic RAG. Real customers ask messy compound questions: 'I want running shoes under $150 — and what's your return policy if they don't fit?' This needs two retrieval types: SQL RAG for the product filter and Document Loading for the returns policy. Give the agent both tools plus your database schema, and it will invoke the product filter for the shoe search, the document loader for the policy, and combine both into one coherent answer.
The tradeoff is latency and non-determinism. For a support chatbot, test whether response speed stays acceptable. If most questions are single-intent, route them directly with the decision tree and reserve Agentic RAG for genuinely compound queries.
When should I add Hybrid Search?
When customers search product catalogues using both brand names and vague descriptors — 'Nike running shoes comfortable for distance.' Full-Text Search nails 'Nike' but misses 'comfortable' matching 'cushioned midsole.' Vector Search catches the meaning but ranks keywords poorly. Hybrid Search runs both in parallel and merges results via Reciprocal Rank Fusion. It's the recommended default when you're unsure, covering both precision and breadth.
Next step: Map your top 10 customer questions to data shape (documents, database, or both) and question type (lookup, filter, semantic, or compound). Run each through the decision tree, then validate against the failure modes — especially confirming predefined SQL queries on your customer-facing surface. Start with the simplest strategy that works.
// FREQUENTLY ASKED QUESTIONS
Should my support chatbot use dynamic SQL queries?
No — avoid dynamic LLM-generated SQL on customer-facing chatbots because they're non-deterministic and harder to control. Use predefined parameterised queries where the structure is fixed and only values change. Extract parameters like price, category, and rating from the customer's question with an LLM, inject them into your template, and execute. Reserve dynamic SQL for internal analytics tooling.
How many policy documents can Document Loading handle?
For a handful of documents, load the full content directly. As your policy set grows, switch to the index sub-strategy — maintain an index of filenames and descriptions so an LLM selects the right document before loading. If the document set becomes large and undifferentiated, consider Vector Search to retrieve targeted passages instead of whole documents.
Will Agentic RAG slow down my support responses?
Potentially, yes. Agentic RAG makes multiple tool-selection decisions and may retry on failure, adding latency. For a customer-facing chatbot, test response times carefully. If most questions are single-intent, route them directly through the decision tree and reserve Agentic RAG only for genuinely compound questions that span policy documents and product data.