Frequently Asked Questions About Edureka MCP-RAG Agentic AI Build Framework
22 answers covering everything from basics to advanced usage.
// Basics
What is chunking and why can't I just feed the whole document to the LLM?
Chunking splits large documents into smaller token-sized units before embedding. Feeding an entire large document at once overloads the LLM, causes it to miss context or hallucinate, and can exceed token limits. Chunking lets the model process content in batches and improves retrieval precision, since queries match against focused chunks rather than sprawling documents.
What is context injection in a RAG pipeline?
Context injection is prepending retrieved document chunks into the LLM prompt before generation. The formula is Final LLM Prompt = User Query + Retrieved Context. This grounds the LLM's answer in your specific data, turning it from a guesser into a synthesizer. Skipping injection and sending only the query produces generic, hallucinated answers.
What is the difference between short-term and long-term memory in an agent?
Short-term memory exists only within an active chat session and is lost when the session ends — like a browser tab you close. Long-term memory is stored in a vector DB or retrieved from historical data and persists across sessions. RAG agents rely on long-term memory via the vector DB, so never depend on session memory for critical knowledge.
What is the ReAct agent pattern?
ReAct (Reasoning + Action) is the agentic reasoning cycle: Perception (understand the problem), Planning (break it into subtasks), Action Execution (run steps and call tools), and Observation (evaluate and realign). The LLM reasons about which tool to call, calls it, observes the result, and either finalizes the answer or iterates — enabling multi-step reasoning.
// How To
How do I set up the Python environment for a RAG agent?
Create a virtual environment with python -m venv <env_name>, activate it, then install from requirements.txt: anthropic (or openai/google-generativeai), chromadb, sentence-transformers, mcp, requests, numpy, and python-dotenv. Always activate the venv before installing — otherwise packages land in your global Python and cause version conflicts and deployment failures.
How do I build the rag.py pipeline?
Structure rag.py in five sub-steps: load documents line-by-line from the knowledge hub and strip whitespace/stop-words; select the MiniLM-L6-v2 embedding model; initialize a ChromaDB client; on first run (collection count is 0), encode all chunks and add them; then define a retrieve() function that embeds the query, calls collection.query with n_results=K, and returns the joined top-K chunks.
How do I build an external tool server with FastMCP?
Create a separate server file per data source. Use FastMCP to define a decorated tool endpoint that accepts input parameters (like a city name), calls the external API with requests.get, and returns a structured response. Add exception handling in every tool server. Each tool becomes an independently hostable MCP server the supervisor agent can invoke.
How do I write an effective context-setting system prompt?
State four things explicitly: the agent's role, the instruction to retrieve context from the knowledge base, the instruction to call the appropriate tool, and the instruction to answer clearly based on the user question. Example: 'You are an AI agent. Retrieve the context. Call the tool. Understand the user question and answer clearly.' Vague prompts produce generic, hallucinated output.
How do I connect the agent to a tool server in app.py?
In app.py (the supervisor agent), define async tool-calling functions using MCP StdioServerParameters pointing to the tool server .py file. In the agent(question) function, set a context variable from retrieve(question), detect keywords to trigger tool calls (like a city name triggering call_weather), construct a system prompt with context plus tool output, send it to the LLM, and return the response.
// Troubleshooting
My agent keeps giving generic answers instead of using my documents. What's wrong?
You're likely skipping context injection — sending only the user query to the LLM without the retrieved RAG chunks. Confirm retrieve() actually returns chunks, that they're prepended to the prompt (User Query + Retrieved Context), and that your system prompt constrains the agent to answer only from the knowledge base. Also verify the collection was populated on first run.
Why does my agent crash when the API fails?
You lack retry and fallback logic. Wrap all API and tool calls in try/except, add retry loops of 2-3 attempts for empty or malformed LLM responses, and define documented fallback messages like 'I do not have sufficient information on this topic.' Log errors with stage labels — chunking, embedding, retrieval, LLM call — so failures are traceable.
My agent says 'insufficient information' for questions it should answer. How do I fix it?
First confirm this isn't correct behavior — non-injected topics should return that response. If the topic IS in your knowledge base, check that documents were embedded and added to ChromaDB, that Top-K isn't too low to catch the relevant chunk, and that your chunking strategy didn't split the answer awkwardly. Try increasing K from 3 to 5 or switching to semantic chunking.
My installed packages aren't found when I run the agent. What happened?
You almost certainly installed packages before activating the virtual environment, so they went into your global Python. Activate the venv first (Scripts/activate on Windows), verify site-packages is populated inside the venv, then reinstall from requirements.txt. Never install outside an activated venv — it causes version conflicts and deployment failures.
// Comparisons
How does MCP compare to using Flask or FastAPI for tool hosting?
Flask and FastAPI host single-purpose web APIs; MCP is purpose-built as the agentic microservice layer for multi-agent, multi-tool orchestration under one interface. MCP standardizes how agents discover and call tools, databases, and data sources, whereas with Flask/FastAPI you'd hand-wire each integration. Think of MCP as Flask for agents — the environment hub connecting reasoning to external capabilities.
How does a single monolithic agent compare to a supervisor with sub-agents?
A single agent handling multiple domains produces routing errors and hallucination as it struggles to disambiguate contexts. A supervisor agent orchestrating one agent per domain goal routes queries cleanly to the right sub-collection or tool, aggregates responses, and returns a synthesized answer. For multi-domain use cases like a hospital with multiple departments, always use the supervisor pattern.
How does a portable embedding model compare to a proprietary one?
A portable model like MiniLM-L6-v2 keeps your embeddings independent of any LLM vendor, so switching providers requires no re-embedding. A proprietary embedding like OpenAI Embeddings locks you in — switch LLMs and you must re-embed your entire knowledge base, a costly and slow migration. For long-lived vector DBs, portability wins.
How does RAG compare to just increasing the LLM's context window?
Stuffing entire documents into a large context window is expensive per token, slow, and dilutes attention across irrelevant content, increasing error. RAG retrieves only the Top-K most relevant chunks, keeping context precise, cheap, and focused. RAG also scales to knowledge bases far larger than any context window and persists knowledge in a vector DB across sessions.
// Advanced
Which chunking strategy should I choose for my documents?
Use Fixed-Size Chunking (e.g., 1000-token windows) as a reliable default. Choose Semantic Chunking when a document has distinct topic sections you want grouped together. Choose Recursive Chunking to bind paragraphs discussing the same sub-topic. Always remove stop-words and normalize text before chunking to reduce token waste and improve retrieval precision.
How do I objectively evaluate my RAG agent's quality?
Run four checkpoints: did it retrieve the right chunks, did it retrieve the correct Top-K count, was relevant context injected before generation, and is the response accurate? Measure accuracy with ROUGE, BERTScore, or BLEU rather than subjective reading. Also verify hallucination is reduced — answers should be specific to injected knowledge, not generic internet-level responses.
How do I build an agent that combines document Q&A with live API data?
Build two sources under MCP: a FastMCP tool server wrapping the live API and a RAG pipeline ingesting documents into ChromaDB. In app.py, detect query intent by keyword — ticker or city keywords trigger the tool server, product or policy keywords trigger retrieve(). Combine both context sources in the LLM prompt and instruct the system prompt to distinguish live data from document knowledge.
Can I route different departments or domains to separate ChromaDB collections?
Yes. Give each department its own knowledge hub folder and ingest it into a separate ChromaDB collection. The supervisor agent detects the domain keyword in the query, routes to the relevant sub-collection, retrieves Top-K chunks via semantic similarity, and injects them. This prevents cross-domain contamination and keeps each agent constrained to its specialist records.
Do I need a fancy UI to run and test the agent?
No. A console is sufficient for testing and iteration. With the venv activated, run python app.py — the agent initializes, loads the knowledge base into ChromaDB, and enters an interactive while-loop prompting for input, breaking on 'exit'. Test both knowledge-base questions and external tool queries before investing in any UI layer.