How ML Engineers Fine-Tune Coding Agents with SFT

Why should ML engineers use agent-driven SFT?

As an ML engineer, you already know how to write a TRL script — but doing it repeatedly across parameter sweeps, wiring up Tracchio logging, scheduling HF Jobs, and managing masking correctly for every run is tedious and error-prone. The agent-driven SFT framework moves you one abstraction level up: you specify outcomes and constraints in a Contract Prompt and verify artifacts, while a coding agent handles implementation. Your skills repo encodes tested TRL and HF Jobs patterns so the agent avoids the failure routes you'd otherwise debug by hand.

This is ideal when you have real agent traces from your product — say tool-call sessions from an internal assistant — and want to focus a small open model like a 2B instruction-tuned model on that behaviour without standing up full RL infrastructure.

How do you set up the pipeline?

Start by collecting agentic traces in multi-turn conversation format: user instruction, assistant tool calls, tool results, continuation. Scrub them with Presidio HF to remove API tokens and credentials, then push the cleaned dataset to HF Hub.

Next, build the skills repo. Create skill files for the SFT workflow (stages, data format, `-100` masking), TRL usage (SFTConfig and SFTTrainer patterns), HF Jobs (scheduling, smoke-testing, retrieving results), and Tracchio observability. Install what you can via the TRL or HF CLI, and add a skill entry for any recurring agent mistake.

Then write the Contract Prompt. It must specify the base model HF ID, dataset HF ID, the SFT algorithm, sweep ranges for learning rate, LoRA rank and sequence length, the HF Jobs hardware tier and budget ceiling, one shared Tracchio project name, per-run adapter push targets, the selection rule (best held-out eval loss), and the eval benchmarks. Require orientation before spending.

How do you keep the run safe and on budget?

Enforce the orientation phase. Before any paid HF Jobs launch, the agent must resolve the exact model ID, confirm the dataset shape, verify push and job authorizations, and pass a cheap smoke-test run without OOM. If it skips this, intervene and add an explicit orientation rule to the Contract Prompt.

During the sweep, monitor Tracchio: train loss decreasing, eval loss tracking train loss, token accuracy rising, entropy falling, and the learning rate schedule tapering. Verify failed runs are diagnosed and retried, not silently dropped — every combination must appear in the same project for fair comparison.

How do you validate and ship the result?

Read the agent-generated script and confirm four things: it applies the chat template via `apply_chat_template`, masks user-side tokens with `-100`, logs to the correct Tracchio project, and pushes each adapter. The agent selects the best run by held-out loss — don't override this manually without a specific reason — then pushes full weights for the winner.

Run evals via HF Jobs using Inspect AI: at least one domain benchmark (held-out task accuracy) and one general benchmark like HumanEval or MBPP to protect general capability. Remember that held-out loss measures imitation quality, not agent capability — SFT gives you the best imitator of your traces, not a superhuman agent.

Finally, save the agent's orchestration trace, review it for wasted compute, and update your skills files. This is how the pipeline compounds into your team's best-known SFT workflow.

Next step: Collect and scrub a small batch of your product's agent traces, stand up a minimal skills repo, and run one smoke-tested sweep on a 2B model to validate the pipeline end to end before scaling.