Intellipaat Agentic AI Systems Builder

// When should you use the Intellipaat Agentic AI Systems Builder methodology?

Use this skill whenever you need to design, evaluate, or build an agentic AI system, RAG pipeline, or LLM-powered application — or when advising a team on whether to use generative AI at all versus a simpler solution.

// What information do you need before designing an agentic AI system?

• Use-case descriptionrequired
  What business problem or workflow should the AI system address?
• Data sourcesrequired
  What internal or external data the system needs to access (HR database, internet, documents, etc.)
• Deployment contextrequired
  Is this a startup, regulated enterprise (banking/finance), or mid-market company? Affects API vs. local LLM decision.
• Compliance constraints
  Are there data-privacy, PII, or regulatory restrictions (e.g., banking/GDPR)?
• Budget signal
  Rough monthly budget or sensitivity to infrastructure cost.

// What core principles guide building production-grade agentic AI systems?

// How do you build an agentic AI system step by step?

1. 1

   Qualify whether generative AI is the right tool at all

   Ask: can this problem be solved with a simpler ML system? If yes, recommend that instead. Generative AI is energy-intensive and expensive. Only proceed if the use-case genuinely requires unstructured input/output, reasoning, or language understanding.

2. 2

   Classify the system type: Generative AI vs. Agentic AI

   Generative AI = prompt-based, single-turn, content creation (text/image/audio/video). Agentic AI = goal-based, multi-step, autonomous task execution with tool use. Determine which category the use-case belongs to before designing anything.

3. 3

   Make the API vs. local LLM deployment decision

   Default to API (e.g., Gemini, OpenAI) unless: (a) the organisation is in a regulated industry (banking, finance, healthcare with strict compliance), or (b) data-sovereignty requirements mandate on-premises. Remember the 32x cost multiplier for local deployment. Always check: input token cost, output token cost, context window size, and knowledge cutoff date before selecting a model.

4. 4

   Select and configure the LLM, noting its four key parameters

   For any model chosen, document: (1) input token price per 1M tokens, (2) output token price per 1M tokens, (3) context window size (input + output tokens combined cannot exceed this), (4) knowledge cutoff date. Set a max_token / max_length limit in all production calls to control cost and behaviour.

5. 5

   Decide whether the system needs RAG (Retrieval Augmented Generation)

   RAG is required when: (a) the LLM's knowledge cutoff makes it unaware of recent events the use-case needs, or (b) the system must answer questions about internal/proprietary data (company HR, documents, policies). If neither condition holds, a direct LLM call may suffice.

6. 6

   Design the RAG architecture with guardrails

   RAG flow: user query → convert to embedding → vector lookup against knowledge base → retrieve relevant chunks → pass (query + retrieved context) to LLM → LLM generates a structured, natural-language answer. CRITICAL: before building, define (a) what data scopes are accessible, (b) what query types are in/out of scope, (c) PII fields that must never be returned. Implement guardrails as explicit filters — not as LLM prompting alone.

7. 7

   Build the agentic layer using LangChain + LangGraph with ReAct pattern

   Use LangGraph as the primary framework (most customisable, production-ready, used by major organisations). Use LangChain for prompt templates, memory, and chain modularity. Implement agents as ReAct agents (Reason → Act loops). Do not use PAL agents — not used in production. Start with a single-agent prototype before building multi-agent systems.

8. 8

   Implement tool integrations and MCP for inter-system communication

   Any external tool the agent must use (search, database, API) should be registered as a tool in LangGraph. For structured communication between AI systems and external tools, implement Model Context Protocol (MCP). Build your own async MCP server if you need a custom node. Avoid Agent-to-Agent (A2A) protocol.

9. 9

   Address hallucination risks explicitly in the system design

   Identify all output types where hallucination would cause business harm (financial figures, dates, names, calculations). For these, add: (a) retrieval verification steps that ground the answer in source data, (b) confidence or source-citation requirements in the prompt, (c) human-in-the-loop checkpoints where the stakes are high enough. Never assume the LLM's confident answer is correct.

10. 10

    Fine-tune the baseline model if the use-case requires specialised output

    If the general pre-trained model does not perform adequately (e.g., domain-specific document fraud detection, specialised Q&A), enter the adaptation phase: fine-tune on labelled domain data (e.g., fraudulent bank statements, salary slips, PAN cards). Fine-tuning is expensive and time-consuming — only do it when API + RAG + prompt engineering cannot close the accuracy gap.

11. 11

    Monitor token consumption and cost in production

    Track input + output tokens per session. Set per-session and daily limits. Use pay-as-you-go pricing models for production systems where token limits are unpredictable. Cost, not just accuracy, determines whether a generative AI system is viable at scale.

// What do real-world agentic AI system designs look like?

// What mistakes should you avoid when building agentic AI systems?

• Treating generative AI as a magical pill — defaulting to it for every problem without evaluating whether a simpler ML system would work and cost less.
• Ignoring the 32x cost multiplier between API and local LLM deployment when making architecture recommendations to a business.
• Confusing context window with total token budget — the context window is the per-conversation limit (input + output combined); exceeding it causes silent truncation, not an error.
• Building a RAG application without guardrails — the system will answer any question including leaking PII, which is a major compliance and reputational failure.
• Assuming LLM output is correct because it sounds confident — hallucination is present in all major models (GPT, Claude, Gemini, Grok) and must be architecturally mitigated, not assumed away.
• Using PAL agents or A2A protocol in production — these are not used in industry; use ReAct agents and MCP respectively.
• Choosing an LLM without checking its knowledge cutoff date — if the use-case requires knowledge of recent events, the cutoff makes the model silently wrong.
• Building with code-generation tools (Cursor, Lovable, Replit, Base44) for security-critical or production-grade software — these tools do not produce secure, production-ready code.
• Skipping the fine-tuning phase when the use-case is highly specialised — a general pre-trained model will underperform on domain-specific tasks like fraud detection without adaptation.
• Giving RAG access to all company data by default — data scope and access permissions must be explicitly designed by humans; the AI will not self-restrict.

// What are the key terms in agentic AI and RAG systems?

Agentic AI

AI systems that act as autonomous agents — setting goals, breaking complex tasks into steps, making decisions, using external tools, and executing actions end-to-end with minimal human input. Distinguished from Generative AI by being goal-based rather than prompt-based.

Generative AI

AI that takes unstructured input (text, image, audio, video, code) and generates unstructured output in those same modalities. It is prompt-based and reactive — it only acts when instructed. It never outputs probabilities, class labels, or regression numbers.

ReAct Agent

The dominant production agentic pattern: Reason + Act. The agent reasons about the task internally, then acts based on that reasoning. Used by LangGraph, CrewAI, NIM, and most industry frameworks. The recommended pattern over PAL agents.

Context Window

The maximum total number of tokens — input tokens (prompt + conversation history) plus output tokens (model response) — that an LLM can process in a single conversation. Exceeding this limit causes the oldest tokens to be silently dropped.

Knowledge Cutoff

The date after which an LLM has no training data. The model cannot answer questions about events after this date unless augmented with RAG or real-time tool access.

Hallucination

When an LLM generates a wrong answer with complete confidence, presenting it as correct. Described as 'the cancer of LLMs.' Present in all major models. Must be mitigated architecturally in any production system.

RAG (Retrieval Augmented Generation)

An architecture pattern where a user query is converted to an embedding, matched against an external knowledge base (internal database or internet), the retrieved context is combined with the query, passed to the LLM, and the LLM generates a grounded, natural-language response.

Guardrails

Explicit architectural rules that restrict what a RAG or agentic system can access, respond to, and return. Guardrails define in-scope query types, blocked data fields (PII), and response boundaries. They are human-designed — not delegated to the AI.

Pre-training Phase

The first phase of LLM development: the baseline model is trained on massive amounts of mostly unstructured data to learn contextual relationships, token probabilities, word order, and general patterns. Requires billions to trillions of tokens.

Adaptation Phase (Fine-tuning)

The second phase of LLM development: the baseline model is specialised on smaller, task-specific labelled data to perform a particular function (e.g., Q&A, image generation, fraud detection). Fine-tuning is expensive and should only be done when API + RAG + prompt engineering cannot close the accuracy gap.

RLHF (Reinforcement Learning from Human Feedback)

The labelling methodology used during LLM training where human annotators label data in multiple-choice question format, providing correct and incorrect answer signals that the model learns from iteratively.

Token

The fundamental unit of LLM input/output. Roughly corresponds to 4 characters of English text (a rule of thumb, not a guarantee). Individual words or sub-word fragments — determined by Byte Pair Encoding (BPE). Both input and output tokens are charged and count toward the context window.

Byte Pair Encoding (BPE)

The encoding strategy used by most LLMs to split text into tokens. Longer or rare words may be split into sub-word tokens, which is why token counts are slightly higher than word counts.

MCP (Model Context Protocol)

The production-grade protocol for structured communication between AI systems and external tools or nodes. The recommended standard over Agent-to-Agent (A2A) protocol, which is considered a short-lived trend.

LangGraph

The recommended primary agentic framework for beginners and production use. Allows building highly customisable graph-based agents. Part of the LangChain ecosystem and used by major organisations.

LangChain

A software library (wrapper) that provides modularity for building any AI application — including agentic pipelines, RAG systems, and LLM integrations. Provides pre-built chains, prompt templates, memory, and tool connectors.

32x Cost Rule

The empirical cost multiplier observed in production: deploying a local LLM on your own server costs approximately 32 times more than using an equivalent API, due to GPU, infrastructure, and maintenance costs.

Pay-as-you-go

The API pricing model where you pay only for the tokens you consume. Recommended for most organisations as the cost-effective default, with token limits set per-session to control spend.

// FREQUENTLY ASKED QUESTIONS