From surprise to rule to skill: how an agent contract teaches itself¶
If you open this project for the first time, the thing that hits you is the governance. CLAUDE.md is thousands of words of imperative HARD RULEs. There is a wall of lints that run on every pull request. There is an entire document, the overview docs, whose only job is to explain why the rules exist. A reasonable first reaction is "who sat down and designed all of this up front?"
Nobody did. That is the interesting part.
None of the contract was designed in advance. Each rule is a fossil. It marks the exact spot where an autonomous agent tripped over something, the failure got caught in a written record, and somebody decided the lesson was worth promoting so it could never recur silently. The contract is thick because the system is built to convert pain into enforcement -- and the conversion happens on a ladder, where each rung makes the lesson harder to forget than the last.
This post walks that ladder, rung by rung, grounded entirely in this repo's real machinery. If you are trying to build a self-correcting discipline into your own AI workflow, the ladder is the transferable part. The specific lints are not; the escalation pattern is.
The ladder¶
Here is the whole shape before we climb it. A lesson starts as a one-off note an agent leaves behind. If it matters, it climbs:
- A surprise note in a run record -- local, easy to miss.
- A follow-up ticket that names the gap as real work.
- A retro that spots the same shape across several run records.
- A CLAUDE.md hard rule plus an enforcing lint -- global, and mechanical.
- A durable skill the provider loads on its own whenever the relevant task recurs -- ambient.
A note is local. A rule is global. A lint is mechanical. A skill is ambient. Each rung trades a little authoring effort for a lot less reliance on the agent choosing to remember. That trade is the entire idea.
Rung 1: the surprise note¶
Every ticket in this project leaves behind a run record -- a *.run.md sibling next to the ticket file, its format pinned in the run-record contract. The run record captures what the diff cannot: the things the ticket did not anticipate. Two of its sections are where lessons are born. ## Surprises / judgment calls is for spec gaps and decisions the agent had to make where the ticket was silent. ## Dead ends is a negative-result log -- what failed first, and why.
These are cheap and honest, and that is exactly the problem with them. They are local. A surprise noted in one ticket's run record is invisible to the next agent working a different ticket, who will cheerfully rediscover the same wall.
The memory-to-skills migration is a clean example of a surprise that mattered. Its run record (the ticket archive) has a ## Surprises / judgment calls bullet noting that writing under the local agent configuration was blocked by Claude Code's hard-coded write rail even though the contract claimed the directory was exempt -- so the agent routed file creation through Python I/O instead. That is a real, specific, reusable lesson. On rung 1 it is also a needle in a haystack of run records.
Rung 2: the follow-up ticket¶
The first promotion is turning a note into work. When an agent notices a gap that is out of scope for the ticket it is on, the rule is not "fix it inline" -- it is "stop, note it, and file a follow-up ticket." A surprise that becomes a ticket has a name, a slug, and a place in the backlog. It is no longer trapped inside one run record; it is a thing the orchestrator can schedule.
This project takes the follow-up so seriously that there is a hard rule about wiring it up. CLAUDE.md's "A fix ticket MUST wire its dependency into every existing ticket it unblocks" exists because filing the fix is only half the job: if the fix is not encoded into the ## Depends on of the tickets it unblocks, the identical failure just repeats on the next batch. The memory-to-skills ticket itself is a follow-up of this kind -- it was born when an audit noticed that feedback files only loaded if an agent thought to read MEMORY.md first, and it deliberately carried a forward-looking note about the review-surface ticket that should come after it.
A ticket is more durable than a note. But it is still one lesson, treated as one task. The leap to a general rule needs something that reads across many tickets at once.
Rung 3: the retro that finds the pattern¶
That something is the retrospective. After a phase or a feedback cluster finishes, a retro runs last (templates: the workflow docs and its phase sibling). Its job is not to write code. Its job is to read every sibling run record -- every ## Surprises / judgment calls, every ## Dead ends -- find the cross-ticket patterns, and decide what to do with each one.
The feedback retro is structured around four questions, and the third is the load-bearing one for this post: candidates for new load-bearing agent rules. For each candidate the retro must either promote it -- by editing the agent-rule files in that same PR -- or reject it with a one-sentence rationale. Every loose end from the cluster's run records gets an explicit disposition: resolved-in-cycle, seed-forward, yagni-closed, or re-scoped. Nothing is allowed to silently drop.
This is the rung where "this annoyed one agent once" becomes "this is a pattern several agents hit." The retro is the pump that lifts a lesson out of the run records. It is also the single point of failure for the whole ladder, which is a thread we will pick back up at the end. The /dev-retro command is what runs it.
Rung 4: the hard rule and the lint that makes it bite¶
When the retro promotes a lesson, it usually lands in two places at once, and the pairing is the point.
First, imperative rule text in CLAUDE.md -- a HARD RULE the agent reads as instruction. Second, a lint registered in review.always_checks in the review configuration, which fails the pull request at review time if the rule is violated. The rule is what an honest agent follows. The lint is what catches a forgetful one. Prose alone trusts the agent to remember; prose plus a gate does not have to.
The bug-fix gates are the clearest worked examples. The rule "On bug-fix tickets, the code commit must land before any test commit" is enforced by lint-bug-commit-order. The rule "Bug-fix tickets require a live-repro evidence artifact" is enforced by lint-bug-repro-evidence. The rule that gameplay randomness must route through RandomnessService is enforced by lint-no-direct-random, which scans the gameplay trees and fails on a raw import random. In each case the imperative text and the mechanical check ship together, because either one alone is leaky: a rule with no lint is a suggestion, and a lint with no rule is a mystery when it fires.
You can watch a rule get born this way. The catalog once renamed its public, no-auth role from SPECTATOR to OBSERVER, and the rename left roughly 160 stale references scattered across the docs even after the first code PR merged. the overview docs records that mess as the origin story of the CLAUDE.md rule "Canonical doc edits propagate through every related doc in the same turn." A surprise -- a rename that did not finish -- became a standing rule that now binds every future concept change. That is rung 3 handing off to rung 4 in one motion.
Rung 5: the durable skill¶
The top rung is the one that fires without anyone deciding to consult it.
For a long time, hard-won tooling lessons in this repo lived as feedback memory files under the local agent configuration. They had the same weakness as a surprise note, one level up: they only loaded if an agent first read MEMORY.md and followed the breadcrumb. The fix was the memory-to-skills migration (the ticket archive), which converted each feedback file into a provider-loaded skill at the local agent configuration. A skill auto-loads whenever the harness matches the current task against the skill's description -- the rule fires even when the agent would never have thought to go looking for it. Same content, ambient recall.
The skill retro-seed-verification is a single lesson that climbed the entire ladder. It started as a retro finding: a cluster audit discovered that transcribing a stale carry-forward seed into the next cluster's plan produced "ghost tickets" -- work to fix bugs that other commits had already fixed. That finding became a feedback memory file. The migration then graduated it into the local agent configuration, which now loads on its own whenever an agent is about to transcribe a retrospective seed and tells it to re-verify the claim against current code first. A surprise about ghost work is now an ambient reflex.
Why the history doc is the spine¶
The ladder only works because the why is preserved somewhere durable, and that is the explicit job of the overview docs. It states its own contract up front: the rule files are authoritative for what an agent must do; the history doc is authoritative for why a rule exists and what must not be lost when the rule text gets shortened. It is an incident index -- the PR clusters, the stale-vocabulary sweeps, the evidence gaps, the workflow-drift incidents that made each terse rule load-bearing.
This matters more than it looks. A rule with no preserved rationale is the thing a future maintainer deletes because it seems fussy, and then rediscovers the original failure the hard way. The history doc is what lets the contract be audited instead of merely obeyed: an agent rewriting or retiring a rule reads the incident behind it first. The ladder produces rules; the history doc keeps them legible.
The meta-lesson: incident-derived, machine-enforced law¶
Step back and the pattern is one idea repeated at every rung. You do not keep an autonomous agent honest by trusting it to remember. You keep it honest by encoding each hard-won lesson where it cannot be skipped.
There are exactly three such places, and the top three rungs map onto them. Imperative rule text the agent reads as instruction. A lint in review.always_checks that fails the pull request whether or not the agent read the rule. A skill the provider loads on a keyword match whether or not the agent went looking. Each is harder to bypass than the last: you can forget to read a long rules file, but you cannot merge past a failing lint, and you cannot fail to notice a skill that has already loaded itself into your context.
That is why the contract grows. It is not bureaucratic accretion. Every rule is incident-derived -- it traces back to a real failure in a real run record -- and machine-enforced -- it bites at review time or load time, not on the honor system. The system is built to convert pain into enforcement, and the ladder is the conversion mechanism.
The honest edges¶
I would be selling you something if I stopped at "and the ladder climbs itself." It does not, and it is not free.
Every rung costs something to author and to maintain. A rule is words in a file that is already very long, and every rule added dilutes the signal of the others -- which is exactly why the retro is required to reject candidate rules with a rationale rather than promote every annoyance. A lint is code that has to be kept in lockstep with the rule it guards: this repo has a whole hard rule, "Process-contract changes must land atomically," precisely because changing one enforced check means updating the config entry, the runner, the help text, and a guard test in the same PR, or the lint silently drifts from the rule. A lint that drifts is worse than no lint, because it enforces a shape the rule no longer means.
Skills go stale too. A skill describes the world as it was when the lesson landed. The irony is sharp here: retro-seed-verification exists because stale carry-forward produces ghost work -- and a skill is itself a piece of carried-forward guidance that can rot if the thing it describes moves. The top rung buys you ambient recall at the price of ambient staleness.
And the whole ratchet stalls the instant retro discipline lapses. Rungs 1 and 2 happen as a side effect of normal work -- agents leave run records, agents file follow-ups. But the lift from a scattered pile of notes to a general rule depends entirely on someone running the retro, reading the run records, and choosing to promote. Skip it and the surprises keep accumulating in individual *.run.md files, unread, and the contract stops learning. The thick CLAUDE.md is not evidence that the system runs itself; it is evidence that someone kept turning the crank. This is the same dependency the compounding-flywheel capstone describes for catalog quality, and the spiral-detection watchdog is a concrete guardrail that came up this very ladder -- a retro-driven mechanism that catches the residual cases the rules do not. It is governance, not magic.
That is the honest version of the answer to "who designed all this?" Nobody designed it, and nobody can stop maintaining it. The contract is a record of every place the system has learned, and it keeps learning for exactly as long as someone keeps reading the run records and deciding which surprises deserve to become law.