Frequently Asked Questions About Karpathy Self-Improving AI Knowledge Base
20 answers covering everything from basics to advanced usage.
// Basics
How long does it take to set up the Karpathy AI knowledge base?
Initial setup takes 30-60 minutes: 10 minutes for folder architecture, 15-20 minutes to write and refine the Claude MD with AI assistance, and 10-15 minutes to dump existing material into Raw. The Wiki build step takes another 30 minutes of AI processing time (you can walk away). Total active human time on day one is about 45 minutes. The system then improves passively every time you dump new material or ask a question.
What kind of files work best in the Raw folder?
Markdown files work best because the AI can parse them cleanly. Use Obsidian's web clipper to convert web articles to markdown in one click. Plain text, meeting transcripts, and book highlights all work well. PDFs are supported but expect lower fidelity — the AI may miss formatting, tables, or embedded images. Screenshots with text can be processed if your AI supports vision. Avoid proprietary formats like .docx when possible; convert to markdown first.
What's the difference between Raw, Wiki, and Outputs folders?
Raw is the junk drawer — all incoming material goes here unorganized. The AI processes it; you never sort it. Wiki is the AI-written, AI-maintained organized knowledge layer with one markdown file per topic, cross-linked and indexed. You never edit it by hand. Outputs stores every AI-generated answer, report, gap analysis, and health check result. Outputs feed back into the system to improve future answers, creating the compounding loop.
Do I need coding skills to build this knowledge base?
No coding is required. The entire system runs on markdown files and an AI with file-system access. You create folders, write text files, and paste prompts. On Mac, Xcode (free) lets you create markdown files quickly. On any platform, a simple text editor works. The AI handles all the complex work — indexing, linking, summarizing, and auditing. The only technical requirement is access to an AI tool that can read and write to a local folder or connected workspace.
What AI plan or subscription do I need for this?
You need an AI plan that supports file-system access and extended context windows. Claude Pro or Max with Projects or Cowork mode is the recommended option. Free tiers typically have usage limits that make the Wiki build and health check steps impractical for larger knowledge bases. If you have extensive material (50+ Raw files), plan for a paid tier. Stagger health checks across the month to spread credit usage evenly.
// How To
Can I build multiple knowledge bases on different topics?
Yes — this is built into the architecture. Your top-level second brain folder acts as a container. Each topic gets its own subfolder with its own Raw, Wiki, Outputs folders and its own Claude MD. Knowledge bases remain independent but can be queried together if needed. Stagger monthly health checks across different days to avoid exhausting AI session credits in one sitting.
How do I write a good Claude MD file?
A good Claude MD specifies: the subject and 3-5 themed focus areas, folder roles (Raw as intake, Wiki as AI output, Outputs as reports), ingestion rules for processing new Raw files, wiki rules (one MD per topic, index first, cross-links, anti-AI writing style), output rules (save every answer as a report), health check schedule and seven-stage audit process, and memory file rules. Work with the AI iteratively — paste a draft, ask it to critique and improve, repeat until the schema is precise.
What is the anti-AI writing style guide and how do I create one?
The anti-AI writing style guide is a set of rules that prevent the AI from producing generic, fluffy prose in Wiki articles. To create one, paste Wikipedia's article on AI-generated writing style issues into your AI and prompt: 'Based on this, write a set of rules you must follow to never produce writing that sounds like AI.' The resulting guide gets included in your Claude MD. Without it, Wiki articles accumulate hollow phrasing that degrades readability and trustworthiness over time.
// Troubleshooting
What if my Raw folder has hundreds of files — will the AI get overwhelmed?
Split the initial Wiki build across multiple sessions. The AI processes files sequentially and has context window limits. For large Raw folders (100+ files), batch the ingestion by instructing the AI to process 20-30 files per session. The Claude MD should specify how to mark files as 'processed' so the AI doesn't re-ingest them. The Change Log tracks what's been handled. Plan for a paid or max-tier AI plan if you have extensive material.
What happens if I accidentally edit a Wiki file by hand?
Manual edits to the Wiki introduce drift that compounds over time. The AI assumes it is the sole author of Wiki content and may overwrite your changes during the next health check, or worse, treat your edits as canonical and propagate errors. If you've made manual edits, the safest fix is to move the edited content to Raw as a new note and let the AI re-process and integrate it properly during the next ingestion pass.
What if the AI generates incorrect information in the Wiki?
The monthly health check is designed to catch this. Stage 3 of the audit checks source provenance — claims not backed by a source in Raw get flagged. If you spot an error between health checks, add a correction note to Raw (not the Wiki) and run an ingestion pass. The AI will update the relevant Wiki article using the new source. Never directly edit the Wiki — that breaks the AI-maintained integrity principle.
What's the biggest mistake people make with this system?
Treating day-one output as the final product. The system is deliberately weak at the start because it only has whatever you initially dumped into Raw. People get discouraged by thin, generic early answers and abandon the system. The entire value proposition is compounding — consistent use, regular dumping of new material, and monthly health checks are what transform it from a basic tool into an irreplaceable asset around day 100.
// Comparisons
Can I use ChatGPT instead of Claude for the Karpathy knowledge base?
Yes, but with limitations. The system requires an AI that can read and write local files or a connected folder. Claude with Projects or Cowork mode supports this natively. ChatGPT can work if you use the Code Interpreter to upload and process files, but it lacks persistent file-system access across sessions. You'd need to re-upload context each time, which breaks the compounding loop. Claude's file-system integration makes it the default choice for this framework.
How is this different from just using a regular second brain in Obsidian?
In a regular Obsidian second brain, you are the librarian — you create links, organize notes, and maintain structure. In the Karpathy method, the AI does all of that. You only dump raw material and ask questions. The AI builds the index, writes summaries, discovers cross-links, and audits for gaps monthly. Obsidian can still be used as a capture and viewing layer, but the organizational intelligence is offloaded entirely to the AI.
How does this compare to using NotebookLM from Google?
NotebookLM lets you upload sources and ask questions, but it doesn't maintain a persistent, self-organizing Wiki or run health checks. It's a read-and-query tool, not a compounding system. The Karpathy method creates a living knowledge base that grows with every interaction — the AI writes and maintains organized articles, discovers connections, and audits itself monthly. NotebookLM is better for quick one-off research; the Karpathy method is for long-term knowledge accumulation.
// Advanced
Can I use this for a team knowledge base, not just personal use?
Yes, but the default architecture assumes solo use. For teams, update the Claude MD to acknowledge collaborative inputs and attribute sources to team members. Each contributor dumps material into Raw independently. The AI should tag which team member contributed each source. Be aware that concurrent edits from multiple people can cause conflicts — designate one person as the 'system owner' who runs ingestion passes and health checks to maintain integrity.
How do I know if my knowledge base is actually improving over time?
Track three indicators: the number of Wiki articles and cross-links (visible in the index), the specificity and source density of answers to your queries, and the health check reports. Early queries will produce generic, thin answers. After consistent use and re-ingestion, answers should cite multiple internal sources and surface non-obvious connections. The monthly health check gap report also shrinks over time as coverage expands — this is measurable proof of compounding.
How does the Change Log work and why do I need it?
The Change Log is a markdown file that records when ingestion runs, health checks, and edits last occurred. It serves as the system's memory across sessions. Without it, the AI cannot distinguish new Raw files from already-processed ones, leading to duplicate ingestion or missed content. Each time the AI processes files or runs a health check, it timestamps the action in the Change Log. This is what makes the system stateful despite using a stateless AI.
Can I export or share my knowledge base with someone else?
Yes — because everything is stored as markdown files in a standard folder structure, the entire knowledge base is portable. Zip the domain folder and share it. The recipient needs their own AI with file-system access and the Claude MD file will instruct their AI on how to operate the system. However, the compounding value is tied to ongoing use, so a static export is a snapshot, not the living system. For ongoing collaboration, use the team approach instead.
Can I automate the health checks so I don't forget?
Yes, if your AI environment supports scheduled tasks or automation triggers. Some setups allow you to create recurring prompts or calendar-triggered workflows that initiate the health check automatically. If your AI tool doesn't support automation natively, set a monthly calendar reminder and use a saved prompt template. Stagger checks across different days if you run multiple knowledge bases to avoid exhausting session credits.