Alejandro AO Agentic RAG n8n Build

Last updated: 21 August 2026

Build a fully functional Agentic RAG system in n8n using open-source LLMs and free Hugging Face embeddings, deployable on a VPS, with optional internet-search capability via Firecrawl MCP.

// TL;DR

Agentic RAG in n8n is a document-aware AI agent architecture where an LLM autonomously decides when to search your knowledge base — replacing the hard-coded query-rewrite-then-search pipeline of Manual RAG. You build two workflows: an Ingestion Workflow that chunks and embeds documents into a vector store, and a Retrieval Workflow where an AI agent uses a 'Knowledge Base Search' tool plus optional Firecrawl web search. Use it when you need to query PDFs, HTML, or Excel files via chat or webhook — powered by free Hugging Face embeddings and cheap open LLMs (MiniMax, Kimi, Qwen), deployable on a VPS.

Framework

// When should you use the Agentic RAG n8n build?

Use this skill when you need to design or implement a document-aware AI agent workflow inside n8n — whether starting from scratch or auditing an existing n8n project. Also applicable when migrating from Manual RAG to Agentic RAG architecture.

// What do you need before building Agentic RAG in n8n?

Documents to ingestrequired
PDF, HTML, Excel, or other files whose contents the agent must be able to query. Can be sourced via form upload, email, Google Drive, or any n8n trigger.
Hugging Face API tokenrequired
Access token from huggingface.co with 'read repos' and 'call inference providers' permissions enabled. Used for both free embeddings and open LLM inference.
Embedding model IDrequired
The Hugging Face model ID of a feature-extraction / embedding model (found by filtering huggingface.co/models by task → NLP → Feature Extraction).
Open LLM model IDrequired
The Hugging Face model ID of the chat/instruction model to power the AI agent (e.g. MiniMax M2.7, Kimi K2.6, GLM 5.1, Qwen 3 27B). Filter by 'inference available' on Hugging Face.
Firecrawl API key
API key for Firecrawl MCP server, required only if internet search / web scraping capability is desired for the agent.
Vector store identifierrequired
Name or collection ID for the vector store (in-memory test store or external e.g. Chroma, Qdrant). Must be consistent across ingestion and retrieval workflows.

// What core principles make Agentic RAG in n8n work?

Agentic RAG vs Manual RAG

Manual RAG hard-codes a pipeline: query rewriting → vector search → optional re-ranking → LLM prompt assembly. Agentic RAG replaces all of that with an AI agent that receives a 'Knowledge Base Search' tool and decides autonomously when to call it, how many times, and with what query — because modern LLMs are smart enough to handle this implicitly.

Two-Workflow Architecture

Split the n8n workspace into two separate workflows sharing the same vector store: (1) an Ingestion Workflow triggered by a file event, and (2) a Retrieval Workflow triggered by a chat message or webhook. This separation keeps document loading decoupled from agent conversation.

Embedding Model Consistency

The exact same embedding model node must be attached to both the vector store insert node (ingestion) and the query data tool (retrieval). Mismatching embedding models at query time will produce meaningless vector similarity scores.

OpenAI Node as Universal Router

Use the n8n OpenAI Chat Model node with a custom Base URL (e.g. router.huggingface.co/v1) instead of provider-specific nodes. This gives access to any OpenAI-compatible inference endpoint — Hugging Face inference providers, Ollama, OpenRouter, Cerebras — by simply swapping the base URL and API key credential.

Tool Description as Agent Instruction

The description field on each tool node is what the agent reads to decide whether to invoke that tool. Write it precisely: name the tool, state what data it contains, and give the agent any heuristics it needs (e.g. 'do not attempt to scrape entire Wikipedia pages — you will exceed context').

Open Models Over Closed for Cost

Open models available via Hugging Face inference providers (MiniMax M2.7, Kimi K2.6, GLM 5.1, Qwen 3 27B, Deep Seek V4) are capable of running AI agents and cost dramatically less — often $1–$3 per million output tokens vs $30+ for closed frontier models. Use them unless a specific capability gap requires a closed model.

JSON Workflow as LLM Target

n8n workflows are stored as JSON. This means you can prompt Claude or Codex to generate an entire workflow JSON and paste it directly into n8n (Ctrl+V on a blank canvas). Be specific about which nodes, integrations, and tool descriptions you need when prompting.

Ingestion Complexity is Underrated

Data extraction from PDFs and structured files looks like a small step but is one of the hardest parts of the ETL pipeline. Tables, images, and unusual layouts all require special handling. Prefer specialised extraction products (e.g. Unstructured) for production; use n8n's built-in binary data loader only for prototyping.

Chunking = Text Splitting

Chunking (called 'text splitting' in n8n nodes) controls retrieval quality: similar information in the same chunk is retrieved together and gives the agent complete, coherent context. Character-count splitting is the baseline; semantic / structure-aware chunking (e.g. Unstructured library) is production-grade.

MCP Client Tool for External Capabilities

The n8n MCP Client Tool node connects any MCP-compatible server to the agent's toolset over streamable HTTP. This is how you give the agent capabilities like live web search (Firecrawl MCP) without writing custom integration code.

// How do you build an Agentic RAG system in n8n step by step?

1
Install and launch n8n
Preferred method: Docker on a VPS. Create a named volume (n8n_data), run the Docker image with port 5678 exposed. Alternatively use `npx n8n` for local testing. Set a strong password immediately if the instance is internet-accessible. For VPS access, connect via SSH tunnel or place behind a reverse proxy with auth.
2
Create Hugging Face credentials
Go to huggingface.co → Profile → Access Tokens → New Token. Enable permissions: 'read repos' and 'call inference providers'. Save the token. In n8n, add this as an OpenAI-type credential with Base URL set to router.huggingface.co/v1 — this single credential serves both embeddings and LLM calls.
3
Select and note your embedding model ID
On huggingface.co/models, filter by Task → NLP → Feature Extraction. Copy the full model ID (e.g. 'BAAI/bge-m3'). Record it — you will paste this ID into two separate nodes and they must match exactly.
4
Select and note your open LLM model ID
On huggingface.co/models, filter by 'inference available'. Recommended capable agent models: Kimi K2.6, MiniMax M2.7, GLM 5.1, Deep Seek V4, Qwen 3 27B, Mistral Small, Llama variants. Check pricing via 'compare providers' on the model page. Copy the full model ID.
5
Build the Ingestion Workflow — add a file trigger node
Create a new workflow. Add a trigger node appropriate to your file source: n8n Form (exposes a public HTML upload URL), Gmail (on message received), Google Drive (on file added to folder), or any integration. For the Form trigger, set accepted file types, mark field as required, optionally add Basic Auth if internet-facing.
6
Add and configure the Vector Store insert node
Go to AI → Other AI Nodes → Vector Stores. Choose your store: in-memory Simple Vector Store for prototyping, or Chroma / Qdrant / Pinecone for production. Set operation mode to 'Insert Documents'. Assign a unique store name/ID — record it for use in the Retrieval Workflow.
7
Attach the Embeddings node to the Vector Store insert node
Search for 'Hugging Face Inference' embeddings node. Connect it to the Embeddings input of the vector store node. Paste your embedding model ID. Set provider to 'auto'. Use the Hugging Face API token credential created in Step 2.
8
Attach the Document Loader node to the Vector Store insert node
Add a Document Loader (binary data) node. Set input source to 'all input data', MIME type to auto-detect. Set text splitting to 'Simple' for prototyping (fixed character chunks) or configure a more sophisticated splitter for production. Connect to the Document input of the vector store node.
9
Test the Ingestion Workflow end-to-end
Click 'Execute Step' on the trigger to submit a test file. Then execute the vector store node to verify documents are chunked and stored. Check the output panel — you should see page content items equal to the number of chunks extracted. Do not proceed to Retrieval until ingestion confirms successful output.
10
Build the Retrieval Workflow — add a chat or webhook trigger
Create a second workflow in the same workspace. Use 'Chat Message Received' trigger for a built-in chat UI (generates a public chat URL), or 'Webhook' (POST endpoint) for programmatic access. Add authentication (n8n user auth or Basic Auth) if the endpoint will be internet-accessible.
11
Add and configure the AI Agent node
Go to AI → AI Agent. Write a system message that explicitly names the available tools and provides heuristics (e.g. 'You have access to: a vector search tool for internal documents, and the Firecrawl MCP for internet research. Do not scrape entire Wikipedia pages.'). Connect the trigger to the agent's input.
12
Attach the Chat Model node to the AI Agent
Add an OpenAI Chat Model node. Use the Hugging Face credential from Step 2 (Base URL: router.huggingface.co/v1, API key: your HF token). Paste the open LLM model ID from Step 4. Optionally add custom headers (e.g. billing org header). Alternatively, use the native Hugging Face Inference Model node and paste the model ID directly — but note it does not provide a searchable model list.
13
Add the Knowledge Base Search tool to the AI Agent
Add a Vector Store Query node as a tool. Set operation to 'Retrieve Documents (for Agent)'. Write a clear tool description the agent will read to decide when to invoke it. Set the same store name/ID used in Step 6. Set result limit (4 is a reasonable default). Enable 'include metadata' so the agent can cite page numbers and sources.
14
Attach the same Embeddings node to the Knowledge Base Search tool
This is the most critical consistency requirement: attach an embeddings node with the exact same model ID used in Step 7 to the query tool's embeddings input. Reuse the same node reference if n8n allows it, or create an identical node. Any mismatch here will silently break retrieval quality.
15
Optionally add the Firecrawl MCP Client tool
Add an MCP Client Tool node. Set connection type to 'Streamable HTTP'. Enter the Firecrawl MCP endpoint URL (include API key in the URL if Firecrawl supports it that way, avoiding separate auth config). The agent will automatically discover and invoke Firecrawl's available tools (search, scrape, crawl) for internet queries.
16
Test the Retrieval Workflow via the chat window
Open the chat trigger's test panel. Send a greeting first to verify basic LLM connectivity. Then ask a question that requires document lookup — verify the agent calls the Knowledge Base Search tool in the execution logs. Then ask an internet question — verify the agent calls the Firecrawl MCP tools. Inspect logs for tool call counts and thinking loops.
17
Publish and monitor
Click Publish, name the version. Copy the public chat URL or webhook endpoint. Monitor via the Executions panel — inspect each run, see which tools were called, identify errors. Track failure rate in the statistics dashboard. For production, connect to an external vector store (Chroma, Qdrant) rather than in-memory store.

// What are real-world examples of Agentic RAG in n8n?

A consulting firm wants internal staff to query a repository of client-uploaded PDF reports without exposing raw files.

Ingestion Workflow: trigger on Google Drive file-added event → Document Loader (PDF) → chunking → Hugging Face embeddings → Chroma vector store. Retrieval Workflow: Chat Message trigger with n8n user authentication → AI Agent (open LLM via HF inference) + Knowledge Base Search tool pointing to the same Chroma collection. Staff access the published chat URL; the agent queries only the ingested reports.

A solo operator wants a daily newsletter agent that researches current events and formats a briefing.

No ingestion workflow needed. Retrieval Workflow: Schedule trigger (daily) → AI Agent with Firecrawl MCP tool only → system prompt instructs the agent to research specified topics and return a formatted newsletter. Output node sends result via Gmail or posts to Slack. No vector store required since all content is live-fetched.

A sales team wants to auto-enrich inbound leads by cross-referencing a CRM form submission against both an internal product knowledge base and live company research.

Ingestion Workflow: n8n Form upload of product docs → embeddings → vector store. Retrieval Workflow: Webhook trigger (POST from CRM) → AI Agent with two tools: (1) Knowledge Base Search on product vector store, (2) Firecrawl MCP for live web research on the lead's company. Agent returns structured enrichment JSON; webhook response delivers it back to the CRM.

// What mistakes should you avoid when building Agentic RAG in n8n?

Using different embedding models for ingestion and retrieval — even a minor mismatch silently destroys retrieval quality because vectors live in incompatible spaces.
Leaving the n8n instance internet-accessible without authentication — anyone with the URL can trigger your workflows and consume your API quota.
Using character-count text splitting (Simple splitter) in production — it splits mid-sentence and mid-concept, degrading both retrieval precision and LLM answer quality.
Writing a vague or missing tool description on the Knowledge Base Search node — the agent cannot decide when to call the tool if the description does not clearly state what data the vector store contains.
Instructing the agent (or allowing it) to scrape entire large pages like Wikipedia — this will exhaust the LLM's context window and cause the run to fail or produce truncated answers.
Running the in-memory Simple Vector Store in production — it does not persist across container restarts; use Chroma, Qdrant, or Pinecone for anything beyond demos.
Forgetting to scope the Hugging Face token with 'call inference providers' permission — embeddings will work but LLM inference calls will be rejected.
Clicking 'Execute Step' on individual nodes and assuming the full workflow works — always run the complete workflow end-to-end before publishing to catch inter-node data-passing bugs.
Pasting an AI-generated workflow JSON without specifying exact node types — Claude or Codex may hallucinate node names that do not exist in n8n; always verify each node is real before relying on generated JSON.

// What key terms should you know for Agentic RAG in n8n?

Agentic RAG: A RAG architecture where an AI agent autonomously decides when and how to query the vector store via a tool, rather than a hard-coded pipeline. The agent handles implicit query rewriting and can make multiple searches before answering.
Manual RAG: The original RAG pipeline: explicit sequential steps of query rewriting → embedding → vector search → optional re-ranking → prompt assembly → LLM call. Now considered legacy due to LLM capability improvements.
Ingestion Workflow: The n8n workflow responsible for extracting text from raw documents, chunking it, embedding it with a Hugging Face model, and inserting vectors into the vector store. Runs on file-upload or schedule triggers.
Retrieval Workflow: The n8n workflow that contains the AI agent and its tools. Triggered by chat message or webhook; the agent queries the vector store and/or external tools to answer user questions.
Knowledge Base Search tool: The vector store query node attached as a tool to the AI agent, configured with operation mode 'Retrieve Documents (for Agent)'. The agent calls this autonomously using its own generated queries.
Text Splitting: n8n's term for chunking — dividing extracted document text into smaller pieces before embedding. 'Simple' splitting uses character count; production systems use semantic or structure-aware splitting.
OpenAI Node (as universal router): The n8n OpenAI Chat Model node used with a custom Base URL to route requests to any OpenAI-compatible inference provider (Hugging Face, Ollama, OpenRouter, etc.), not necessarily OpenAI itself.
MCP Client Tool: An n8n tool node that connects to any MCP (Model Context Protocol) server over streamable HTTP, exposing that server's tools directly to the AI agent.
Firecrawl MCP: A Firecrawl-hosted MCP server that gives the AI agent sophisticated internet capabilities: Google search, web page scraping, and site crawling — invoked autonomously by the agent when it needs live information.
Inference Providers: Third-party compute backends accessible via Hugging Face's router (router.huggingface.co/v1) that host open-source models for inference. Examples: Novita, Cerebras. Accessed with a Hugging Face API token that has inference provider permission.
Embedding Model Consistency: The requirement that the exact same embedding model is used at both ingestion time (to embed document chunks) and retrieval time (to embed the agent's search query). Violation causes semantically meaningless similarity scores.
Workflow JSON: n8n's internal representation of a workflow as a JSON file. Can be copied (Ctrl+C on canvas), pasted (Ctrl+V), shared, and generated by LLMs like Claude or Codex for rapid workflow scaffolding.

// FREQUENTLY ASKED QUESTIONS

What is Agentic RAG in n8n?

Agentic RAG in n8n is an architecture where an AI agent is given a 'Knowledge Base Search' tool and autonomously decides when to query your vector store, how many times, and with what query. It replaces Manual RAG's hard-coded pipeline (query rewrite → search → re-rank → prompt) because modern LLMs handle those steps implicitly. You build it as two n8n workflows sharing one vector store.

What's the difference between Agentic RAG and Manual RAG?

Manual RAG hard-codes a fixed pipeline: query rewriting, vector search, optional re-ranking, then prompt assembly. Agentic RAG hands the AI agent a search tool and lets it decide when and how to search — including making multiple searches before answering. Agentic RAG is now preferred because modern open LLMs are smart enough to handle query rewriting and retrieval decisions implicitly.

How do I build an Agentic RAG system in n8n step by step?

Install n8n (Docker on a VPS), create Hugging Face credentials with inference permissions, then build two workflows. The Ingestion Workflow: file trigger → Document Loader → embeddings → vector store insert. The Retrieval Workflow: chat/webhook trigger → AI Agent → Chat Model + Knowledge Base Search tool (same embedding model) + optional Firecrawl MCP. Test each end-to-end, then publish.

How do I get free embeddings for RAG?

Use Hugging Face Inference Providers. Create a token at huggingface.co with 'read repos' and 'call inference providers' permissions, then pick a feature-extraction model (filter models by Task → NLP → Feature Extraction, e.g. BAAI/bge-m3). Attach the Hugging Face Inference embeddings node in n8n. The same token also powers open LLM inference — one credential serves both.

How does using open LLMs compare to using GPT-4 or Claude for RAG agents?

Open models via Hugging Face inference providers (MiniMax M2.7, Kimi K2.6, GLM 5.1, Qwen 3 27B, Deep Seek V4) cost roughly $1–$3 per million output tokens versus $30+ for closed frontier models — a 10x+ savings. They're fully capable of running RAG agents. Use open models unless a specific capability gap forces a closed frontier model.

When should I use Agentic RAG instead of a simple chatbot?

Use Agentic RAG when the agent must answer questions grounded in your own documents (PDFs, HTML, Excel) rather than just its training data. It's also ideal when the agent needs to decide dynamically whether to search internal docs, do live web research via Firecrawl, or both. If you only need live web research and no documents, skip ingestion and use a Firecrawl-only agent.

Do I need a vector store to build Agentic RAG in n8n?

Yes, if the agent must query documents — the vector store holds your embedded chunks. Use the in-memory Simple Vector Store only for prototyping; it doesn't persist across container restarts. For production, use Chroma, Qdrant, or Pinecone. However, if your agent only does live web research via Firecrawl MCP with no document lookup, you don't need a vector store at all.

How do I give my n8n agent internet search capability?

Add an MCP Client Tool node set to 'Streamable HTTP' and point it at the Firecrawl MCP server endpoint (include your Firecrawl API key in the URL). The agent automatically discovers Firecrawl's tools — search, scrape, and crawl — and invokes them when it needs live information. Instruct it in the system prompt not to scrape entire large pages like Wikipedia to avoid context overflow.

Why is my RAG retrieval returning irrelevant results?

The most common cause is embedding model mismatch — using a different embedding model at ingestion than at retrieval. Vectors then live in incompatible spaces and similarity scores become meaningless. Attach the exact same embedding model ID to both the vector store insert node and the Knowledge Base Search query tool. Other causes: poor chunking (character-count splitting) or a vague tool description.

What results can I expect from an Agentic RAG n8n build?

A published chat URL or webhook endpoint where an AI agent answers questions grounded in your documents, cites page numbers when metadata is enabled, and optionally does live web research. Costs run $1–$3 per million tokens on open models with free embeddings. You'll monitor tool calls and failure rates in the Executions panel and can scale from in-memory prototype to Chroma/Qdrant production.

Can I generate an n8n workflow with Claude or ChatGPT?

Yes — n8n workflows are stored as JSON, so you can prompt Claude or Codex to generate the full workflow JSON and paste it onto a blank canvas with Ctrl+V. Be specific about exact node types, integrations, and tool descriptions. Always verify each generated node actually exists in n8n, since LLMs may hallucinate node names that don't exist.

Can I run Agentic RAG in n8n on a VPS?

Yes. Run n8n via Docker on a VPS: create a named volume (n8n_data), run the image with port 5678 exposed, and set a strong password immediately. Access it via SSH tunnel or place it behind a reverse proxy with authentication. Never leave an internet-accessible instance without auth, or anyone with the URL can trigger workflows and burn your API quota.

// GET THIS SKILL — FREE