How to Fine-Tune an AI That Writes in Your Voice

Why can't prompts make AI sound like me?

Because prompt engineering can guide a model but cannot escape the behaviours baked in during its original training. You've probably written a long system prompt describing your tone — direct, jargon-free, concise — only to watch the AI drift back to verbose, generic phrasing. That's the prompt ceiling. No amount of prompt iteration overcomes what the base model learned during pretraining. To make an AI reliably sound like you, the behaviour has to be baked into the model's weights through fine-tuning.

When should a consultant fine-tune instead of using RAG?

Run the decision ladder first. Try a better prompt. If that fails, add RAG so the model can retrieve your past work. If RAG still produces off-voice output, try an agentic loop. Only when all three fail — the output still doesn't sound like you — is fine-tuning warranted. For voice and style specifically, fine-tuning usually wins fast, because tone is an embedded behaviour, not a retrievable fact. RAG can hand the model your emails, but it can't make the model write like them.

How do I turn my emails into training data?

Your corpus is everything you've already written: emails, reports, proposals. Gather it and count your tokens. Then comes the most consequential step — dataset engineering. Clean every artefact: spelling slips, inconsistent formatting, stray punctuation. The model cannot self-correct dirty data; whatever you feed it gets baked in permanently.

Next, transform those raw documents into chat-format prompt-response pairs. Raw emails aren't usable as-is. Use a local language model to generate plausible client questions for each passage, then pair the passage as the ideal answer. This mirrors how you'll actually query the model at inference time — a client asks something, and it responds in your voice.

How do I train and test it?

Train a LoRA adapter targeting roughly 0.5–1.5% of the model's parameters. You never retrain billions of weights — the adapter injects your voice cheaply. Use an Nvidia GPU with CUDA; ensure the model fits in dedicated VRAM rather than relying on RAM offloading, which stalls training. Budget 2–3 hours per run and validate your parameters on a short test job first.

Then evaluate. Send the same client-style questions to both the vanilla base model and your fine-tuned model. Score for stylistic match — does yours sound like you while the base sounds generic? If the fine-tuned model misses, the fix is almost always in your dataset or LoRA parameters, not in post-hoc prompting.

Finally, export the merged model to GGUF and run it in LM Studio or Ollama on your own machine, keeping client data local.

What's my next step?

Start by pulling together every piece of client-facing writing you've produced and estimating your token count. If you're under roughly a million tokens, focus on cleanliness and consistency over volume. Run the decision ladder honestly first — if a sharper prompt or RAG solves it, you've saved yourself a weekend. If not, you now have a clear, repeatable path to an AI that finally sounds like you.