Frequently Asked Questions About Karpathy Self-Improving AI Knowledge Base
22 answers covering everything from basics to advanced usage.
// Basics
What exactly is a 'second brain' in this context?
A second brain is a personal knowledge base external to your own memory where you hold all your information, make connections between ideas, and use it to inform decisions and output. In this system, the second brain is a top-level folder containing one or more AI-maintained knowledge bases, each with a Raw junk drawer, an AI-written Wiki, and an Outputs folder.
What is the Raw folder and why is it called a junk drawer?
Raw is the capture folder where all incoming material — articles, notes, screenshots, transcripts, PDFs — is dropped unorganised and unedited. It's a junk drawer by design because sorting is the AI's job, not yours. Making Raw pretty wastes time and defeats the purpose. Just dump everything in as markdown files and let the AI process it.
What is the compounding loop and how does it work?
The compounding loop is the feedback cycle where every answer the AI generates gets saved back into Outputs and eventually ingested into the system. Each question makes the next answer better because prior outputs and gap reports feed future queries and health checks. This is what makes the system grow from basic on day one to a genuine asset by day 100.
What inputs do I need to start building this?
You need three things: a subject or domain focus (e.g. investing, marketing strategy), existing raw knowledge material (articles, notes, transcripts, PDFs), and an AI environment with file-system access like Claude Cowork. Optionally, add an anti-AI writing style guide and three to five themed focus areas to tune the Claude MD for deeper coverage.
// How To
How do I quickly get web articles into the Raw folder?
Use the Obsidian web clipper browser extension, which converts any web page to a clean markdown file in one click. On Mac, Xcode (free) lets you create markdown files fast via File > New from Template > Markdown File. You can also paste content directly into the AI chat and instruct it to save each item as an MD file in Raw.
How do I write a good Claude MD schema file?
Specify seven things: the subject and themed focus areas; folder roles; ingestion rules for processing Raw; wiki rules (one MD per topic, index first, cross-links, anti-AI style); output rules (every Q&A generates a saved report); the health check schedule and seven-stage audit; and memory file rules. Work iteratively with the AI to refine it before ingesting any content.
How do I query the knowledge base and find my gaps?
Start a new session, point the AI at the folder, and ask a domain question — the AI reads the index, pulls relevant Wiki entries, and answers with sources. Save that report to Outputs. Then ask: 'Based on everything in the Wiki, what are the three biggest gaps in my understanding of this topic?' Save that gap report to Outputs too.
How do I run the monthly health check?
Once a month, prompt the AI to run a seven-stage audit across the Wiki: contradictions, broken backlinks, unsourced claims, coverage gaps, stale articles over 90 days, new article candidates, and undrawn connections. The AI files a health check report in Outputs and updates the Change Log. In interactive mode, choose which findings to action and the AI drafts and ingests accordingly.
How long does the initial setup and Wiki build take?
Dumping existing material into Raw should take 10–15 minutes for most people. The initial Wiki build takes around 30 minutes of AI processing time — you point it at the folder, give the build prompt, and walk away. Writing and refining the Claude MD takes an iterative session. Long jobs can be split across multiple sittings if credits run low.
// Troubleshooting
Why is my Wiki full of generic AI-sounding prose?
You likely skipped or ignored the anti-AI writing style guide. Generate one by pasting Wikipedia's AI writing style article into the AI and asking it to write rules to never do any of that. Reference those rules in your Claude MD so the AI applies them during Wiki builds. Without it, articles accumulate generic prose that degrades readability and trust.
Why is the AI re-processing files it already ingested?
You're probably missing a Change Log or memory file. The Change Log doubles as the system's memory, recording when ingestion and health checks last ran so the AI knows what is new versus already processed. Without it, the AI can't tell what's changed in Raw, leading to duplicate work or missed ingestion. Add one and reference it in the Claude MD.
Why do my outputs feel inconsistent between sessions?
Your Claude MD is likely under-specified or out of date. Without a precise schema, the AI has no consistent instruction layer, so its behaviour drifts between sessions. Tighten the Claude MD to cover ingestion rules, wiki rules, output rules, and the health check process. Work with the AI iteratively to improve the file before relying on it.
The Wiki build keeps running out of session credits — what do I do?
The Wiki build and health checks are credit-intensive, so split the work across multiple sessions or use a paid/max plan. Don't try to build the entire knowledge base in one sitting. For health checks across multiple knowledge bases, stagger them on different days of the month so credits are spread out rather than exhausted at once.
My PDFs aren't being parsed well — is that normal?
Yes, PDFs are harder for the AI to parse and typically produce lower fidelity than markdown. Use them if that's what you have, but expect gaps. Where possible, convert content to clean markdown first — the Obsidian web clipper handles web pages well, and pasting text directly into the chat to be saved as MD gives the best results.
// Comparisons
How does this compare to a traditional RAG or vector database setup?
Unlike RAG systems that require embeddings, chunking, and a vector database, this approach has the LLM maintain a plain markdown index and read whichever files it needs on demand. It's simpler to set up, fully transparent (everything is human-readable markdown), and requires no infrastructure — just an AI with file-system access. You trade some retrieval scale for radical simplicity.
How is this different from just dumping notes into Obsidian?
Obsidian is a manual tool — you write, link, and organise notes yourself. This system inverts that: the AI does all the librarian work, cross-linking topics, auditing for gaps, and drafting new articles. You can even use Obsidian's web clipper to feed Raw, but the organisation, maintenance, and compounding intelligence come from the AI, not you.
How does this differ from asking a general AI chatbot my questions?
A general chatbot answers from public training data and forgets everything between sessions. This system answers from your curated material, cites your own sources, and remembers via the Change Log. Crucially, it compounds — every answer and gap report is saved back and improves future responses, so it becomes a personalised asset a generic chatbot can never be.
// Advanced
Can I use a team version of this instead of solo?
Yes, but the default Karpathy architecture assumes solo use, so you must update the Claude MD to acknowledge collaborative inputs and attribute sources to specific team members. Dump team research reports, saved articles, and notes into Raw, and let the AI build a shared Wiki. Add attribution rules so the AI credits who contributed each piece of source material.
How do I automate the monthly health check?
If your AI environment supports scheduled tasks, set the health check to run automatically once a month. For multiple knowledge bases, schedule each on a different day so session credits are spread across the month rather than consumed all at once. The automated check files its report in Outputs and updates the Change Log without manual intervention.
What is guided ingestion mode and when should I use it?
Guided ingestion mode is an optional interactive process where the AI walks you through ingesting new material step by step, rather than you silently dumping files. Use it when you want more control over how specific items are categorised, when onboarding complex or ambiguous content, or when training the system on a new sub-theme you want handled deliberately.
How do I structure multiple independent knowledge bases correctly?
Create a top-level second brain container folder, and inside it give each subject its own self-contained folder with Raw, Wiki, Outputs, and its own Claude MD. Optionally add a top-level Claude MD describing the container structure. The bases stay independent but can be queried together. Point the AI at the container to ask cross-domain questions across all of them.
What does the seven-stage health check audit actually check?
It checks for: (1) contradictions and inconsistent data between articles, (2) broken backlinks and orphaned references, (3) source provenance for claims not backed by Raw, (4) coverage gaps relative to Raw content, (5) stale articles older than 90 days, (6) suggested new article candidates from gaps, and (7) suggested connections between articles not yet drawn. The AI reports and, in interactive mode, drafts fixes.