Frequently Asked Questions About Intellipaat Agentic AI Systems Builder
22 answers covering everything from basics to advanced usage.
// Basics
What does 'unstructured in, unstructured out' mean for LLMs?
It means generative AI takes unstructured input — text, image, audio, video, or code — and produces unstructured output in those same four modalities. It never natively outputs probabilities, class labels, or regression numbers. Any claim that an LLM directly produces those violates fundamentals. If you need structured output, you must deliberately engineer the coercion from unstructured text into a consistent format.
What are the two phases of an LLM's lifecycle?
Every LLM has a pre-training phase and an adaptation phase. Pre-training learns patterns from massive unstructured data — billions to trillions of tokens — capturing token probabilities, word order, and context. Adaptation (fine-tuning) specialises the baseline model on smaller, task-specific labelled data for a function like Q&A, image generation, or fraud detection. Knowing which phase your problem lives in tells you whether fine-tuning is required.
What is a token and how is it different from a word?
A token is the fundamental unit of LLM input and output, roughly four characters of English text as a rule of thumb. Tokens are individual words or sub-word fragments determined by Byte Pair Encoding (BPE), which is why token counts run slightly higher than word counts — longer or rare words get split. Both input and output tokens are charged and count toward the context window.
What is the difference between a ReAct agent and a PAL agent?
A ReAct (Reason + Act) agent reasons about a task internally, then acts based on that reasoning, looping between the two. It's the dominant production pattern used by LangGraph, CrewAI, and NIM. PAL agents are not used in industry. For any production agentic system, default to ReAct — it's what modern frameworks implement.
// How To
How do I configure an LLM before using it in production?
Document four parameters for any model you choose: input token price per 1M tokens, output token price per 1M tokens, context window size (input plus output combined can't exceed it), and knowledge cutoff date. Then set a max_token or max_length limit in every production call to control cost and behaviour. Skipping the cutoff check is a common way to ship a silently wrong system.
How do I build a RAG pipeline end to end?
Take the user query, convert it to an embedding, run a vector lookup against your knowledge base, retrieve relevant chunks, then pass the query plus retrieved context to the LLM to generate a grounded answer. Before building, define accessible data scopes, in- and out-of-scope query types, and blocked PII fields. Implement guardrails as explicit filters, not prompt instructions alone.
How do I decide whether to fine-tune a model?
Only fine-tune when API plus RAG plus prompt engineering cannot close the accuracy gap. Fine-tuning is expensive and time-consuming, so reserve it for highly specialised tasks like document fraud detection, where a general pre-trained model won't reach 98%+ accuracy. In that case, train on labelled domain data — for example, fraudulent and genuine bank statements, salary slips, and IDs across all document types.
How do I build a multi-agent content pipeline in LangGraph?
Build a graph with one ReAct agent per node: a research agent, a script-writer agent, a validator agent, and a scheduler agent. Give each node access to relevant tools — web search, a document writer, a calendar API — registered in LangGraph. Chain them sequentially. Add a validation node that checks outputs against source material to reduce hallucinated facts. At this scale, cost is typically well under $4/month on API.
How do I monitor and control token cost in production?
Track input plus output tokens per session, then set per-session and daily limits. Use pay-as-you-go pricing, where you pay only for tokens consumed, since token usage is often unpredictable. Cost — not just accuracy — determines whether a generative AI system is viable at scale, so treat token monitoring as a first-class production requirement, not an afterthought.
// Troubleshooting
My RAG system is returning off-topic answers and leaking data — what went wrong?
You almost certainly built RAG without guardrails, so it answers any question — including leaking PII, phone numbers, or off-topic responses. Fix it by explicitly defining accessible data scopes, valid query types, and PII fields that must never be returned, then implementing those as filters in the architecture. Prompting the LLM to 'be careful' is not a guardrail; the AI will not self-restrict.
My conversation is silently losing earlier context — why?
You've exceeded the context window, which is the combined input-plus-output token limit per conversation. When you cross it, the oldest tokens are silently dropped rather than raising an error, so the model appears to 'forget' earlier turns. Track both sides of the budget, trim or summarise conversation history, and set max_token limits to keep total tokens under the window.
The LLM gave a confident answer about a recent event that turned out wrong — why?
The model's knowledge cutoff makes it unaware of anything after its training date, so it either hallucinates or answers from stale data, both with full confidence. If your use-case needs recent knowledge, you must augment the model with RAG or real-time tool access. Always check the cutoff date before selecting a model — mismatched cutoff is a silent correctness failure.
My local LLM deployment is blowing the budget — did I make the right call?
Probably not, unless you're in a regulated industry or bound by data-sovereignty rules. Local deployment costs roughly 32x more than an equivalent API because of GPU, infrastructure, and maintenance. Since 80–90% of real-world use cases run on API, reconsider whether your compliance constraints genuinely mandate on-premises. If they don't, migrating to a pay-as-you-go API is the standard fix.
// Comparisons
How does generative AI compare to a traditional ML classifier for structured problems?
For structured problems that output probabilities, class labels, or regression numbers, a traditional ML system is usually cheaper, faster, and more appropriate. Generative AI is designed for unstructured input and output and is energy-intensive and expensive. The methodology's first step is to qualify this: if a simpler ML system solves the problem, use it. Defaulting to generative AI everywhere is the 'magical pill' pitfall.
How does LangGraph compare to LangChain?
LangChain is a software library providing modularity — pre-built chains, prompt templates, memory, and tool connectors — for building any AI application. LangGraph is a graph-based agentic framework built in the LangChain ecosystem, recommended as the primary tool for production agents because it's highly customisable. In practice you use both: LangGraph for the agent graph and orchestration, LangChain for prompt templates, memory, and chain modularity.
How does MCP compare to Agent-to-Agent (A2A) protocol?
MCP (Model Context Protocol) is the production-grade standard for structured communication between AI systems and external tools, and it's what you should build on. A2A is considered a passing trend. If you're choosing a protocol for tool and inter-system communication, invest in MCP — build your own async MCP server for custom nodes rather than adopting A2A.
Should I use code-generation tools like Cursor or Replit for production AI systems?
Not for security-critical or production-grade software. Tools like Cursor, Lovable, Replit, and Base44 do not produce secure, production-ready code. They're useful for prototyping and exploration, but for systems handling sensitive data or business-critical logic, treat their output as a draft requiring real engineering review, not a shippable product.
// Advanced
When is a direct LLM call better than building a full RAG or agentic system?
A direct LLM call suffices when the use-case is single-turn content creation, needs no internal or proprietary data, and doesn't depend on knowledge after the model's cutoff. If you don't need retrieval, multi-step reasoning, or tool use, adding RAG or an agentic layer just increases cost and complexity. Match architecture to the actual requirement rather than over-engineering.
How do I handle a use-case that mixes classification and generation, like document fraud detection?
Recognise that fraud detection is fundamentally a classification or anomaly-detection workflow enhanced by generative AI, not a RAG problem. Deploy locally if compliance forbids sharing documents with APIs, fine-tune on labelled fraudulent and genuine documents to reach high accuracy, and add human-review checkpoints for borderline cases, since a false positive harms real customers. RAG is not the primary pattern here.
Why is structured output described as 'where the money is'?
Because LLMs natively produce unstructured text, but real systems need consistent, structured output specific to the user or system requirement. Coercing unstructured LLM output into a reliable structured format is non-trivial and must be engineered deliberately — through schemas, validation, and parsing. That engineering is the hard, valuable part of building production generative AI, which is why it's framed as where the commercial value lives.
How should I sequence a single-agent prototype into a multi-agent system?
Start with a single-agent ReAct prototype to validate the core loop, tool integrations, and output quality before adding complexity. Once the single agent reliably performs its task, decompose the workflow into specialised nodes — each a ReAct agent with its own tools — and orchestrate them in a LangGraph graph. Building multi-agent first invites debugging chaos; prove the unit before scaling the system.
What is RLHF and where does it fit in LLM training?
RLHF (Reinforcement Learning from Human Feedback) is a labelling methodology used during training where human annotators label data in multiple-choice format, providing correct and incorrect answer signals the model learns from iteratively. It shapes the model's behaviour during development. As a systems builder you generally consume RLHF-trained models via API rather than performing RLHF yourself, but understanding it clarifies why models behave as they do.