Fine-Tuning Agents on Harness Traces for Researchers

Why is agent-driven SFT useful for research?

If you study agent training, SFT is the cleanest way to isolate imitation dynamics: the model learns to reproduce the assistant completions in a trace dataset and nothing more. This framework makes those experiments repeatable and observable. You define the experiment as a Contract Prompt, and a coding agent handles the mechanics — orientation, script authoring, sweep scheduling, and eval — while you focus on the research question. Because every run logs to one Tracchio project, you get comparable curves across hyperparameters for free.

A canonical research use case: training a small model to follow a specific agent harness's tool-call format using publicly available traces from that harness.

How do you encode a harness format as a training constraint?

Locate the public trace dataset on HF Hub and verify it's scrubbed of secrets. In your SFT workflow skill file, specify the target harness's chat template format as an explicit constraint. The agent's generated script applies it via `apply_chat_template` and masks all user-turn tokens with `-100` labels, so the loss is computed only on the harness's assistant-side tool-call language.

Because agentic traces are long, sweep sequence length aggressively — truncation choices materially change what the model can learn about multi-step tool-call chains. Include it in the sweep alongside learning rate and LoRA rank. This makes sequence length a first-class hyperparameter, which is often the most interesting variable in harness-imitation studies.

How do you measure convergence and imitation quality?

Use a harness-specific eval as your primary benchmark: can the fine-tuned model actually complete tasks in that harness? Run it via HF Jobs using Inspect AI. In Tracchio, monitor entropy decrease as evidence the model is converging on the harness's tool-call distribution, alongside falling train and eval loss and rising token accuracy.

Crucially, treat held-out loss as a measure of imitation quality, not agent capability. SFT produces the best imitator of the traces — the run with the lowest held-out loss — which is the correct selection criterion but not a claim about autonomous capability. This distinction is central to any rigorous SFT study: don't over-interpret low loss as proof of a capable agent.

How does this fit into the broader training stack?

SFT is step one — imitation. RL and environment-based training are required for genuine capability improvement beyond the demonstrations. The framework makes this boundary experimentally tractable: you can hold the SFT stage constant and vary trace quality, sequence length, or LoRA rank to study how imitation scales, then compare against RL baselines.

After each run, save the full agent orchestration trace as an HF dataset. It's both a debugging artifact and, potentially, training material for future experiments. Update your skills files with any failure routes the agent took, so the pipeline becomes more reliable across a research programme.

Next step: Pick a public harness trace dataset, write a Contract Prompt that pins the harness chat template and sweeps sequence length, and run a smoke-tested sweep to establish an imitation baseline before comparing against RL.