Skip to content

What an AI researcher wants from a dev loop

A coding-agent run is more useful to research when it keeps more than the final patch. The final patch says what survived. It does not say what the model knew, which local rule it used, which command failed, or which review gate caught the first version.

This repo is building around a smaller unit: context, decision, outcome. The AI audit trail docs call that the trajectory triple. In implementation, it shows up as ticket prompts, run records, trajectory JSONL, audit-store blob pointers, and history-index tables such as trajectories, lint_gate_vector, run_rewards, grounding, and preference_pairs.

That shape maps cleanly to the things researchers ask for: demonstrations, ranked choices, verifiable rewards, process labels, and replayable eval episodes.

SFT wants the steps, not only the answer

Supervised fine-tuning, or SFT, learns from demonstrations. For a coding agent, the weak demonstration is:

ticket -> final diff

That is missing the behavior the next agent needs to copy. A useful demonstration includes the local context and the next good move:

ticket + rules + files read + tool results -> next good action -> verified outcome

For this project, the good action is often procedural. Read the ticket's ## Context files before editing. Use bash scripts/dev.sh setup when the environment is missing instead of searching the filesystem for another binary. Edit canonical workflow/docs/, not generated site-src/. When workflow docs change, mirror the equivalent surface into tempo/. When a ticket requires a run record, write the sibling *.run.md before exiting.

None of those are universal Python or React rules. They are repo-local behavior. SFT can learn that behavior only if the example preserves the context that made the choice correct and the outcome that proved it.

Preference learning wants the fork in the road

Preference learning needs comparable alternatives from the same situation. Direct Preference Optimization, or DPO, is one common way to train from those pairs: choose response A over response B.

Agent development creates these contrasts all the time:

  • one run edits generated staging docs; another edits the canonical source;
  • one run skips the required run record; another writes it before exit;
  • one implementation invents a fixture shape; review replaces it with a real server-shaped fixture;
  • one ticket author leaves a dependency as prose; another encodes it in ## Depends on.

The final branch hides many of those contrasts because review fixes the survivor. The audit trail keeps a place for them. preference_pair trajectory events store a rejected artifact pointer, a chosen artifact pointer, and the review-correction text. The history index flattens those events into preference_pairs rows.

There is an important current limit: preference-pair emission is a manual audit CLI path, not an automatic review-stage guarantee. The project has the schema and collector. It should not pretend every review correction is already a training pair.

RLVR wants checks the repo can run

Reinforcement learning with verifiable rewards, or RLVR, needs rewards that an environment can check. In this repo, many rewards are already executable:

  • Did the exact verification commands exit 0?
  • Did lint-doc-tags accept the edited docs?
  • Did lint-devblog-mirror confirm the Blog and DevBlog share slug and H1?
  • Did lint-run-records parse the required run receipt?
  • Did a bug ticket include the required evidence artifact?
  • Did a prompt-builder change update its golden snapshot?

The audit trail does not need to invent a grader for those. Review records lint_result trajectory events for the configured .review.always_checks. The history index turns them into lint_gate_vector rows with gate order, exit code, pass, and trip flags.

The reward surface is more than pass or fail. run_rewards joins known acceptance exit codes, first-pass review approval, review iteration count, and post-merge stability. Stability is recomputed from local history: a bug follow-up ticket that depends on the original ticket, or a revert commit that cites it, can mark the run unstable inside the configured window.

That is a better reward than "green at the end." It distinguishes a run that passed every local rule on the first review from a run that reached the same final patch only after avoidable convention failures.

Process rewards want the middle

A process-reward model scores intermediate steps. That matters because agent runs often go wrong before the final diff exists.

Good process is visible here:

  • reading the ticket and its context files before editing;
  • using the project wrapper instead of a raw tool path;
  • running the ticket's verification commands, not a similar command from memory;
  • writing a valid run record with surprises, dead ends, and prevention notes;
  • keeping workflow and Tempo surfaces in sync when workflow contracts change.

Bad process is visible too:

  • probing /usr/bin or ~/.local for a replacement executable;
  • touching generated docs staging instead of the tracked source;
  • inventing a closed-vocabulary capability token;
  • adding a compatibility shim for an internal rename;
  • turning an out-of-scope observation into an opportunistic refactor.

The current audit code gives those steps several queryable hooks. tool_call events can carry files_read and search_queries. The grounding table compares the read set with the gold edit set from a captured patch, or with the ticket's ## Touches when no patch is available. Spiral watchdog labels can attach weak negative process signals to suspicious regions such as repeated tool use.

Those labels are not a complete judge of intent. They are useful because they make the middle inspectable instead of leaving it inside a transient chat log.

Self-hosted eval wants replayable episodes

A self-hosted eval asks whether a model can work inside this repo's operating system. The episode needs enough information to replay the attempt:

base commit
+ ticket
+ project rules
+ files read and tool outcomes
+ patch candidate
+ verification commands
+ review gates
+ reward and history labels

The project already has an audit-eval command for completed tickets. It reads the ticket, sibling run record, base_commit, captured trajectory, reward vector, and historical lint vector. It can apply a candidate patch in an isolated checkout and run the ticket verification commands plus the configured review gates.

That makes model-upgrade testing local. A new model is not better just because it solves a benchmark. It needs to remember that scripts/dev.sh and workflow/scripts/dev.sh are different wrappers, that workflow docs may need Tempo mirrors, that run records are required artifacts, and that missing environment setup is handled through the project setup command rather than filesystem hunting.

The practical requirement

All of this comes down to one requirement: keep the facts while they are still cheap.

The ticket has intent. The run record has the human-readable outcome. Git has the patch. The audit trail keeps the ordered middle. The history index makes the compact surfaces queryable.

SFT, preference learning, RLVR, process-reward models, and self-hosted eval do not need five unrelated logging systems. They need the same run described at different resolutions: what the model knew, what it chose, and what the project could verify afterward.