Frequently Asked Questions About Denisov Agent Specialization RAG vs Fine-Tuning Framework
22 answers covering everything from basics to advanced usage.
// Basics
What does 'garbage in, garbage out' mean for AI agents?
It means both RAG and fine-tuning pipelines are only as good as the data fed into them. Bad source data produces bad retrieval results or a model that has learned your mistakes. In RAG, unclean documents surface wrong chunks; in fine-tuning, unfiltered examples teach errors permanently. Good, cleaned, curated data is the foundation of any specialised agent — even if curation shrinks your dataset.
What are the four core base-model failure modes?
The four failure modes are: (a) hallucinating facts that don't exist, (b) forgetting context over long conversations, (c) returning outdated or irrelevant information, and (d) ignoring system instructions on large context windows. Each points to a different fix — hallucination and outdated info lean toward RAG, while forgotten instructions and behaviour drift lean toward fine-tuning.
What is the Behaviour-Change Gate?
It's the first decision checkpoint: 'Do I need to change the model's tone, style, output format, topic restrictions, or reasoning patterns?' A YES answer mandates fine-tuning for that dimension, because RAG cannot teach behaviour. Only if the answer is NO do you proceed to the Data-Dynamism Gate. This gate prevents the common mistake of trying to fix behaviour with more context.
What is the Data-Dynamism Gate?
It's the second decision checkpoint: 'Is my knowledge data dynamic (changing daily, hourly, or per-product) or static?' Dynamic data mandates RAG, because fine-tuning a model on a product catalog that changes hourly isn't viable. Static data opens the choice to fine-tuning, RAG, or hybrid — at which point you score against the full comparison table.
// How To
How do I build the pre-retrieval pipeline for RAG?
Work through three stages before touching the retriever. First, document processing: parse, clean, and normalise all source data into a consistent format. Second, chunking: choose a strategy — recursive character chunking is the recommended default. Third, indexing: embed chunks into a vector database and attach metadata (country, category, product ID) at index time so you can filter precisely later.
How do I implement Hybrid + Rerank retrieval?
Combine keyword search and semantic vector search to retrieve roughly 100 candidate chunks, then apply a reranker model to surface the top 10 most relevant. This hybrid pattern outperforms pure semantic search alone. For multi-hop queries — like 'I don't like pizza' requiring inference — pre-process the user query to extract explicit search intent before hitting the vector store.
How do I prepare training data for fine-tuning at scale?
Use an agentic data-preparation pipeline: run your source documents through an LLM to generate question-answer pairs automatically. Then generate paraphrased variants of each question (same meaning, different wording) to improve generalisation. Prioritise quality — 1,000 clean pairs beat 1 million garbage ones. Clean and curate before training, and regenerate data from newly resolved tickets rather than retraining daily.
How do I choose a fine-tuning toolchain?
Match it to your model type. For managed API models (Gemini, GPT), use the vendor's fine-tuning endpoint — simplest path, least control, pay per token. For open-source models, use Google Colab or Kaggle (free tier for small datasets) with LoRA SDKs for full control. Factor in your ecosystem: on Google Cloud, Google's tools are cost-efficient; on Azure, use Azure-native equivalents.
How do I choose the right chunking strategy?
Default to recursive character chunking — it's a strong balance for most semi-structured text. Fixed-size is simple but risks mid-sentence splits. Semantic chunking groups by meaning. Agentic chunking lets an LLM decide the optimal split — highest quality but highest cost, requiring an LLM call per chunk. Start simple and only upgrade to agentic chunking when results are demonstrably insufficient.
// Troubleshooting
Why does my agent recommend competitor products despite my system prompt?
Because system-prompt instructions aren't permanent — on long conversations, the model forgets constraints like 'never mention iPhone.' This is a behaviour problem, not a data problem, so RAG won't help. Fine-tune the model to internalise the topic restriction, which locks it into the weights so it cannot be context-flushed even in extended conversations.
Why is my fine-tuned model getting worse after retraining?
You're likely hitting catastrophic forgetting — each new fine-tuning run can overwrite previously learned knowledge and capabilities. Track this explicitly and validate against your prior benchmarks after every retraining cycle. Rather than retraining daily on new tickets, regenerate a clean training set periodically and test that the model still handles tasks it previously did well.
Why does my RAG agent match the wrong entity?
You're probably missing metadata at index time. For structured domains — organisations by country, products by category — failing to attach and filter by metadata forces the model to resolve ambiguity semantically, leading to wrong entity matches. Attach metadata like country, category, and product ID when indexing, then filter on it during retrieval to narrow the candidate set correctly.
My RAG results are bad — should I switch retrievers?
Probably not. Poor retrieval is almost never caused by the retriever itself. The root cause is usually upstream: bad document processing, the wrong chunking strategy, or missing metadata at index time. Audit your pre-retrieval pipeline first. Also verify you're using Hybrid + Rerank rather than pure semantic search before blaming or replacing the retriever component.
// Comparisons
RAG vs fine-tuning: which reduces latency?
Fine-tuning reduces latency because there's no database round-trip at inference time — the knowledge lives in the model weights. RAG adds latency by querying a vector store on each request. If low latency is critical and your knowledge is static, fine-tuning wins. If freshness or citations matter more, accept RAG's latency cost, or use a hybrid to balance both.
How does the hybrid approach compare to using RAG alone?
RAG alone supplies fresh data but can't change behaviour — it won't fix tone, format, or topic restrictions, and system-prompt constraints still get forgotten on long conversations. Hybrid fine-tunes the base model for durable behaviour, then layers RAG for fresh, citation-ready data. For real-world production agents needing both consistent style and current facts, hybrid is the recommended best-of-both-worlds approach.
How does this framework compare to a generic 'just add RAG' approach?
Generic advice defaults everything to RAG, which fails for behaviour problems and misdiagnoses failure modes. This framework diagnoses the specific failure first, then runs two gates to route you correctly. It prevents expensive mistakes like fine-tuning on dynamic data or using RAG to enforce brand voice, and it scales to multi-agent orchestration when one agent shouldn't do everything.
// Advanced
When should I use a prompting-only agent instead of RAG or fine-tuning?
Use prompting-only agents for simple tasks that don't require dynamic data lookups or behavioural specialisation. The framework's 'start simple, then increase complexity' principle means you only escalate to RAG or fine-tuning when prompting is demonstrably insufficient. In multi-agent systems, prompting-only agents often handle the simplest sub-tasks while the orchestrator routes harder queries elsewhere.
How do I design a multi-agent architecture with this framework?
Don't force one agent to do everything. Place a maximally-specialised orchestrator agent in the centre to detect user intent and route requests: prompting-only agents for simple queries, RAG agents for dynamic data lookups, and fine-tuned agents for styled or domain-specific responses. Each agent is specialised only to its sub-task, matching the specialisation method to the specific need.
Should I always pick the top embedding model from the leaderboard?
No. Over-indexing on benchmark rankings ignores ecosystem fit. The best embedding model on a leaderboard may not be optimal given your cloud platform, latency requirements, or per-token cost structure. If you're on Google Cloud, Google's embeddings may be more cost-efficient despite ranking lower. Weigh benchmark performance against real deployment constraints before committing.
How do I handle multi-hop queries in RAG?
Pre-process the user query before hitting the vector store. For queries requiring inference — like 'I don't like pizza,' which implies searching for non-pizza options — extract the explicit search intent first. This query rewriting step converts implicit meaning into a searchable form, dramatically improving retrieval for multi-hop questions that pure vector similarity would otherwise miss.
Can I use one fine-tuned model across multiple tenants?
Yes, and LoRA makes it efficient. Because LoRA freezes the base model's weights and adds only a small adapter file, the same base model can serve multiple LoRA adapters without re-uploading the full model each time. This makes it ideal for multi-tenant use cases and edge deployment, where you swap lightweight adapters rather than loading separate full models.
Is fine-tuning worth it for a small dataset?
Yes, if quality is high — 1,000 clean QA pairs can outperform a million noisy ones. For small datasets, LoRA on a free-tier Colab or Kaggle notebook is cost-effective. The key is curation: clean and paraphrase your examples. Fine-tuning shines when you need durable behaviour like tone or topic restrictions that RAG simply cannot deliver, regardless of dataset size.