Frequently Asked Questions About Website Brain Vault Build Method
21 answers covering everything from basics to advanced usage.
// Basics
What is the difference between a Website Brain and a regular Obsidian vault?
A regular Obsidian vault is a general-purpose collection of markdown notes organized however the user prefers. A Website Brain is a purpose-built vault generated from a scraped website where every page becomes an interlinked note with structured metadata, embedded images, screenshots, and a Design DNA note. The key differentiator is the dense, automated interlinking and AI-readability — it's designed as infrastructure for AI agents, not just human note-taking.
What is Brainstein and how does it work with Claude?
Brainstein is a Claude skill that acts as a domain-expert orchestrator during Brain builds. It dispatches multi-agent parallel tasks (the Multi-Agent Wizard pattern), fetches best practices from the internet, and structures the resulting knowledge into a vault. It works alongside the Claude Obsidian skill — Brainstein handles orchestration and expert knowledge, while Claude Obsidian handles file creation, markdown formatting, and wikilink syntax.
Do I need a paid Firecrawl plan to build a Website Brain?
No, Firecrawl's free tier includes 1,000 credits, which is enough for most small-to-medium websites (40-60 pages). However, for large sites with hundreds or thousands of pages, you'll likely exhaust the free credits before completing the crawl. In that case, upgrade to a paid plan before starting. Always check your remaining credits before initiating a large scrape to avoid an incomplete build.
Can I build a Website Brain for a competitor's site?
Yes, the method works on any publicly accessible website — your own, a client's, or a competitor's. Building a competitor's Website Brain is valuable for content gap analysis, design benchmarking, and SEO competitive research. Layer claude-seo on top to compare their page structure, keyword targeting, and internal linking strategy against your own site's Brain.
// How To
How do I set up the .env file for Firecrawl API keys?
Inside your vault folder, create a plain text file named '.env' (with the leading dot). Add the line FIRECRAWL_API_KEY=your_actual_key_here. Never paste the key directly into Claude's chat window — only reference the .env file path. This keeps your credentials secure, reusable across sessions, and out of conversation logs. You can add additional API keys (DataForSEO, etc.) to the same file on new lines.
How do I check interlinking quality in the Website Brain?
Open Obsidian's Graph View (Ctrl/Cmd+G or from the command palette). A healthy Website Brain shows a dense web of interconnected nodes where every major page connects to related pages. If you see isolated dots scattered across the graph — an 'ADHD Brain' — prompt Claude to audit all page notes and repair missing wikilinks. Every page should reference at least its parent section, sibling pages, and any content it links to on the live site.
How do I run Plan Mode before building the Website Brain?
Switch Claude to Plan Mode using the Max model tier (not ultracode). Submit a prompt that includes: the target URL, the mission to scrape all content and Design DNA, instruction to use Firecrawl, instruction for multi-agent parallel execution, and the .env file path. Let Claude ask clarifying questions — answer them all. Only after the plan is fully confirmed do you switch to ultracode/workflow execution mode to begin the actual build.
How do I connect my Website Brain to an existing Marketing Brain?
After completing the Website Brain build, decide whether to merge vaults or cross-link them. For merging, copy the Website Brain's notes into a subfolder of your Marketing Brain vault and let Claude re-resolve all wikilinks. For cross-linking, keep them as separate vaults but create index notes in each that reference key notes in the other. Merging is recommended for your own business or active clients since it enables single-session cross-referencing.
How do I handle images and screenshots in the Website Brain?
Firecrawl extracts images, SVGs, GIFs, and full-page screenshots during the scrape. The Claude Obsidian skill embeds these directly into their corresponding page notes using Obsidian's ![[image.png]] syntax. All image files are stored inside the vault folder. The Design DNA note specifically catalogs logos, hero imagery, and brand graphics. If any images are missing after the build, run the quality check prompt and Claude will identify and re-fetch gaps.
What model tier should I use for each phase of the Website Brain build?
Use the Max model tier for Plan Mode — this is where Claude asks clarifying questions, defines scope, and proposes the build structure. Never use ultracode for planning. Once the plan is confirmed and locked, switch to ultracode/workflow execution mode for the actual scraping and vault creation. Using the wrong tier at the wrong phase leads to either shallow planning (ultracode for planning) or slow execution (Max for bulk file writing).
// Troubleshooting
What do I do if Firecrawl misses pages during the scrape?
After the build completes, run a quality check prompt: 'Do a full review and confirm all pages are scraped, all images are saved, all internal links are resolved, word counts are present, and Design DNA is documented. Report any gaps.' If pages are missing, provide the specific URLs and prompt Claude to re-scrape only those pages. Common causes include JavaScript-rendered content, gated pages, or hitting the Firecrawl credit limit mid-crawl.
Why does my Website Brain Graph View show disconnected nodes?
Disconnected nodes indicate missing wikilinks — an 'ADHD Brain.' This typically happens when the agent creates page notes but doesn't insert [[wikilinks]] to related pages. Prompt Claude: 'Audit every page note in the vault. For each note, add wikilinks to all pages it references in its content, plus its parent section and sibling pages. Report all links added.' Re-check Graph View afterward — it should show a dense, interconnected web.
What if my site has more than 1,000 pages and I'm on Firecrawl's free tier?
Firecrawl's free tier caps at 1,000 credits, so a 1,000+ page site will require a paid plan. Before starting, estimate total pages using a sitemap or site:domain.com search. If upgrading isn't an option, prioritize crawling the most important sections first (homepage, service pages, top blog posts) and build the Brain incrementally. You can always extend it later when credits become available.
// Comparisons
Can I use a different scraping tool instead of Firecrawl for the Website Brain?
Firecrawl is the recommended tool because it natively outputs markdown, full-page screenshots, brand colors, and component data — all critical for a Website Brain. You could theoretically substitute another scraper, but you'd need to handle markdown conversion, screenshot capture, and Design DNA extraction separately. The workflow is optimized around Firecrawl's API output format, so substituting introduces friction and likely extends build time significantly.
How does a Website Brain compare to using a RAG system for website context?
RAG systems retrieve relevant chunks from embedded documents at query time but lose structural relationships, interlinking patterns, and visual context. A Website Brain preserves the full site architecture — page hierarchy, internal link graph, Design DNA, and embedded screenshots — as navigable markdown. AI agents can traverse the entire knowledge structure rather than relying on similarity-based chunk retrieval. The Brain also persists and compounds with each skill layered on top, while RAG indexes are typically static.
How does building a Website Brain compare to manually briefing an AI on a website?
Manual briefing requires you to summarize content, describe branding, and specify links every time you start a new task — it's slow, incomplete, and non-reusable. A Website Brain captures everything once in a structured, persistent format that any AI agent can reference across unlimited future sessions. It also captures things humans typically forget to brief: exact color hex codes, font stacks, internal link targets, image alt text, and page word counts.
// Advanced
Can I build a Website Brain for a site behind a login or paywall?
Standard Firecrawl scraping works on publicly accessible pages. For sites behind authentication, you'd need to configure Firecrawl with session cookies or authentication headers if their API supports it. Alternatively, you can manually export the gated content to markdown files, place them in the vault folder, and then prompt Claude to integrate and interlink them with the publicly scraped pages. This hybrid approach works but requires more manual effort.
Can I automate Website Brain updates when the site changes?
Yes, you can schedule periodic re-scrapes of the target site and prompt Claude to diff the new content against existing vault notes, updating changed pages and adding new ones. However, use caution — if you're updating your own site's content based on the Brain, avoid mass AI-generated updates that could trigger Google's spam detection. Update pages thoughtfully and incrementally, especially on large sites.
What should I do with the Website Brain after building it?
The Website Brain is a foundation, not a deliverable. Layer additional skills on top: run claude-seo for on-page audits with full site context, run claude-blog for content gap analysis and on-brand post generation, use Codex with the vault open to generate brand-matched images and social posts in parallel, or schedule automated page update tasks. Each subsequent skill run references and enriches the Brain, compounding its value over time.
Is the Website Brain method only for Claude, or can other AI models use the vault?
The vault is plain markdown with wikilinks and embedded images — any AI model that can read files can use it. Claude Code is optimized for the build process because of the Brainstein and Claude Obsidian skills, but the finished vault works with GPT-4, Gemini, or any agent that accepts file context. You can also point Cursor, Windsurf, or other AI-enabled IDEs at the vault folder for code-aware website editing.
Why should I update old website pages before creating new ones?
Google rewards updated, well-structured existing pages over new thin content. If your site has hundreds of outdated posts from 2016 that aren't ranking, updating them with current best-practice structure and high-value keywords yields faster SEO results than publishing new posts. The Website Brain makes this efficient — each old post's current state is already captured with word counts, headings, and links, so Claude can systematically audit and improve them one by one.