How to Build an Agent for Docs Q&A Plus Live Data
For Product and developer teams building customer-facing agents · Based on Edureka MCP-RAG Agentic AI Build Framework
// TL;DR
Product and developer teams can use the MCP-RAG framework to build an agent that answers product-catalog questions from documents while also fetching live data — like stock prices or inventory — through external APIs. Build two sources under MCP: a RAG pipeline ingesting catalog docs into ChromaDB, and a FastMCP tool server wrapping the live API. The agent detects query intent by keyword, routes to retrieve() for document questions or the tool server for live data, and combines both context sources in the LLM prompt. The system prompt instructs it to distinguish live data from document knowledge in its answer.
Why combine RAG with live tool-calling?
Customer-facing agents often need two kinds of knowledge: stable document content like a product catalog, and real-time data like current stock prices or live inventory. A pure RAG agent can't fetch live data, and a pure tool-calling agent can't ground answers in your docs. This framework unifies both through MCP. You get an agent that answers 'What are the specs of model X?' from your catalog and 'What's the current price?' from a live API — in the same conversation, with each source clearly attributed.
How do you build the two data sources?
First, build the RAG pipeline in rag.py: ingest catalog documents into ChromaDB, chunk them, embed with MiniLM-L6-v2, and expose a retrieve() function returning the Top-K relevant chunks. Second, build a FastMCP tool server that wraps the live API — define a decorated endpoint accepting input parameters, call the external API with requests.get, add exception handling, and return a structured response. Each tool server is independently hostable, which is the whole point of the MCP microservice layer.
How does the agent decide which source to use?
In app.py, the supervisor agent function detects query intent. If the question contains ticker or price keywords, it async-calls the stock tool server via MCP StdioServerParameters pointing to the tool file. If it contains product keywords, it calls retrieve() against the vector DB. Both context sources are combined into the LLM prompt.
The system prompt is critical here: instruct the agent to distinguish between live data (tool output) and document knowledge (RAG context) in its answer. This prevents the model from blending real-time figures with static catalog text in confusing ways. Follow the ReAct pattern — the agent perceives the query, plans which source to hit, executes the call, observes the result, and finalizes or iterates.
How do you make it production-ready?
Load the LLM API key from a .env file with python-dotenv — never hardcode it. Wrap every tool invocation and LLM call in try/except with 2-3 retry attempts, and define fallback messages like 'Live data is temporarily unavailable' that double as debugging signals. Log errors with stage labels so you can trace whether a failure happened in retrieval, the tool call, or the LLM. Start Top-K at 3 to keep catalog context precise, and increase only if answers are consistently incomplete.
Because you used a portable embedding model, switching LLM providers later won't force you to re-embed the catalog. And because tools are separate MCP servers, you can add new APIs — currency conversion, shipping estimates — without touching the RAG pipeline.
What results can product teams expect?
Expect a single agent that seamlessly answers catalog questions and live-data queries, clearly attributes each source, degrades gracefully on API failures, and stays grounded to reduce hallucination. The modular MCP design means you scale by adding tool servers, not rewriting the agent.
Next step: Build your rag.py catalog pipeline and one FastMCP tool server, then wire them together in app.py with keyword-based intent routing. Test both query types in the console before shipping.
// FREQUENTLY ASKED QUESTIONS
How does the agent tell a product question from a live-data question?
The agent function in app.py detects query intent by keyword. Ticker or price keywords trigger an async call to the tool server; product or catalog keywords trigger retrieve() against the vector DB. Both results are combined in the LLM prompt, and the system prompt instructs the agent to distinguish live data from document knowledge in its answer.
Can I add more external APIs later without rebuilding everything?
Yes. Each tool is an independent FastMCP server, so you add a new API — currency conversion, shipping estimates — by creating another tool server file and registering it in app.py. The RAG pipeline and existing tools stay untouched. This modularity is the core benefit of the MCP microservice layer.
What happens if the live API is down?
With proper hardening, the agent degrades gracefully instead of crashing. Wrap the tool call in try/except with 2-3 retries and a documented fallback message like 'Live data is temporarily unavailable.' The agent can still answer catalog questions from RAG while signaling the live-data failure clearly to the user.