Read before you write: the habits that keep an AI agent honest¶
If you have paired with a coding agent for more than an afternoon, you have watched it do the confident-and-wrong thing. You ask it to write a test for a screen, and it produces a beautiful test against a state object the real system never emits. You ask it to assert a sequence of events, and it asserts event names that sound right and do not exist. The code compiles. The test is green. The test is also meaningless, and nothing in the diff tells you so.
It is tempting to read that as a model-quality problem -- a smarter model would not have guessed. It is not. The agent guessed because the real answer was not in front of it, and guessing is what any system does when it is asked for a fact it does not have. The fix is not a better model. The fix is a habit: read before you write. Find out what the system actually does -- the real log, the real payload, the real shape -- and only then write the thing that depends on it.
This post is the pattern behind several of the disciplines this project documents one at a time elsewhere. Each of those posts demonstrates one habit in one place. Here I want to pull them into a single portable stance you can carry into your own codebase, with or without any of this project's tooling. It is four habits that compose into one sentence, and one honest caveat about what the habits can and cannot buy you.
Habit 1: read the real log; never guess event names¶
The first habit is the smallest and the most load-bearing. When you need to assert what a system did -- which events fired, in what order, carrying what fields -- do not write those event names from memory. Dump the real ones first.
This project's testing post, The test that knows what it is, names the failure mode exactly: "When you write a timeline assertion, the failure mode is guessing event names. You think the flow emits client.surface_rendered; it actually emits something adjacent; the spec is green against your imagination and red against reality, or worse, green against both and meaningless." A test that is green against your imagination is worse than no test, because it looks like coverage.
The way out is mechanical. The browser exposes a dev/test-only diagnostics surface (window.__bgui), and a Playwright helper, getClientLog(page), reads the same in-memory client log a spec can assert against. The documented loop is discover-then-specify: run the flow against a real backend, read what actually fired, confirm the exact event names and ordering against the closed catalog in the server docs, and only then encode the ordered (and forbidden) events as a named timeline. The guide opens with the rule that makes this possible: "Do not invent new event names without updating this file." The log is a closed vocabulary precisely so that reading it is a reliable substitute for guessing.
The general version, with no window.__bgui in sight: whenever your test or your code asserts a string the system produces -- an event name, a status, a route, a key -- treat that string as something to observe, not something to recall. Recall is where the agent's confidence outruns its information.
Habit 2: drive tests from server-shaped fixtures; never hand-craft state¶
The second habit is the same instinct one layer up. When a test needs an input that the real system would normally produce -- the state a component renders, the payload a handler receives -- get that input from the real producer, not from your imagination of it.
This project makes the instinct a hard rule. CLAUDE.md load-bearing rule 14 reads: "Stage-component tests must not hand-craft UIState objects. UIState test inputs must come from server-shaped fixtures through selectors, recorded event replay, or committed golden UIState fixtures; inline fake UIState object literals fail the project dev wrapper." That rule is the cleanest concrete instance of "read before you write." A hand-crafted state object is a guess wearing the costume of data: it looks exactly like what the server sends, right up until the field you invented diverges from the field the server actually populates, and your test passes against a world that does not exist.
This is the discover-then-specify loop again, pointed at inputs instead of events. You discover the real shape by running a server-shaped fixture through the real transform, and then you specify the test against that shape. You do not author a plausible-looking object from memory and assert against your own fiction. The testing post gives this a name from its taxonomy: an input that came from a recorded session carries the evidence:replay mark -- "a claim about provenance: the input came from a real recorded session, not from an agent's mental model" -- and the evidence:handcrafted mark exists to keep agent-invented inputs visible to a sweep, "exactly where an AI most easily encodes a premise the real system never produces."
The portable form: prefer real producers over hand-built doubles. A fixture captured from the system is a fact; an object you typed that resembles the system is a hypothesis. Tests built on hypotheses pass for the wrong reasons.
Habit 3: state the rule the agent reads, then add a lint that makes it bite¶
The first two habits are things an agent should do. The third is what you do so the agent keeps doing them when you are not watching.
Writing a rule into the agent's instructions is necessary and not sufficient. An instruction is a hope; it competes with every other thing in the context window and loses quietly. The discipline that holds is to pair each rule the agent reads with a check that fails the build when the rule is broken. Rule 14 is not load-bearing because it is written down -- it is load-bearing because lint-no-handcrafted-uistate rejects the inline object literal mechanically. The pattern repeats across the project: gameplay randomness must route through a service, and lint-no-direct-random scans for the forbidden direct import; device profiles must stay device-shaped, and lint-no-personhood watches for person-shaped fields; duplication drift is caught the moment it lands by lint-dedup-drift. The rule states the intent for the human and the agent; the lint is what makes drift a failure instead of a disappointment.
The duplication post, Trimming the duplication an AI cannot see, puts the underlying reason plainly: "You cannot instruction-tune away a missing input." A rule the agent forgot to apply, or never had in view, produces exactly the same drift as no rule at all. A lint does not forget and does not need the rule in its context window, because it re-derives the violation from the code every time it runs.
The portable form: any convention you actually care about should have a mechanical check behind it. If a rule matters and nothing fails when it is violated, the rule is decoration. State it for the reader; enforce it for reality.
Habit 4: the blind spot is the context window, not the model¶
The fourth habit is the frame that explains the other three. An agent does not fail because it is unintelligent. It fails because the relevant fact was not in front of it.
The duplication post says this in a section heading -- "The blind spot is a property of the context window, not the model" -- and the rest of the post is the worked consequence: an agent editing one ticket at a time "does not carry a whole-repo model of every retry wrapper, deep merge, role-string check, formatter, registry, or superseded path you have ever written," so when the helper it needs already exists three modules over, it writes a fresh near-copy that "passes review in isolation." The defect is invisible from inside the ticket. It is only visible from a vantage point -- the whole repo at once -- that no single context window holds. The same horizon limit is what makes a hand-crafted state plausible (the agent cannot see the server schema it contradicts) and a guessed event name confident (the agent cannot see the log catalog it violates).
Read the three earlier habits through this lens and they stop being three separate rules. Reading the real log is putting the event catalog in view. Driving from server-shaped fixtures is putting the real payload shape in view. Adding a lint is putting the rule in view of a process that cannot forget it. Every one of them is the same move: the work is not to make the agent smarter, it is to put the right thing in front of it. Once you see the blind spot as the context window rather than the model, the disciplines stop feeling like a list to memorize and start feeling like instances of one job.
So: read before you write¶
Stack the four and the sentence writes itself. The blind spot is the context window, so the relevant fact has to be in view (habit 4). Putting it in view means reading the real thing first -- the real log (habit 1), the real payload (habit 2) -- rather than recalling a plausible version of it. And the rules that say "read first" only survive contact with a forgetful context window when a lint makes them bite (habit 3). The stance underneath all of it is a refusal to write the thing that depends on a fact until you have actually looked at the fact. Discover, then specify. Read, then write.
It is worth noticing how little of this is specific to AI. A careful human engineer does the same things -- checks the real payload, greps the actual event name, distrusts the test that passed too easily. The reason it has to be explicit with an agent is that the agent has no slowly-accumulated mental index to fall back on; it starts every task with that index empty, the way Device profiles, not user accounts describes an agent autocompleting a person-shaped profile screen from "millions of training examples" because nothing in view told it this product was different. The habit is human; the need to write it down and enforce it is what working alongside an agent makes non-negotiable.
The honest edge: shape is not taste¶
I want to be as honest about the limit here as the disciplines deserve, because overselling them would undo the point.
These habits are not free, and they do not buy as much as they look like they buy. They are not free because they are authoring cost paid up front: dumping the real log before writing the spec is slower than typing the event name you think it is, and capturing a server-shaped fixture is more work than hand-rolling an object that looks close enough. The same boundary shows up in Shipping a React client I have not read: if the real fact is "the AI built this and I have not read it," the honest move is to say that, not to convert the output into a personal learning story. The whole bet is that the up-front cost is cheaper than the silent-wrong-test cost it prevents, and that bet is usually right -- but it is a cost, paid by a human or an agent every time, not a thing you set up once.
And they buy less than they appear to, because a gate can only check shape, not taste. This is the part to internalize. lint-no-handcrafted-uistate can prove a test's input came from a real fixture; it cannot prove the test asserts the thing that actually matters. A timeline spec can be built from real, dumped events and still assert a trivial ordering while the load-bearing event goes unchecked. The duplication finder, as that post is careful to say, "reports confidence bands, not a delete button" -- it ranks candidates and refuses to issue verdicts, because reachability is a fact and "should this be removed" is a judgment. Every gate in this project validates that the input is real and the rule was followed; not one of them validates that the human chose the right thing to test. The discipline protects the seam where AI most easily goes wrong -- inventing inputs and events the system never produces -- and it leaves the harder seam, is this the assertion worth making?, exactly where it has to live: with a person.
That is the honest shape of "read before you write." It is a strong defense against the confident-and-wrong failure, because it forces the agent to look at reality before it commits to a claim. It is not a substitute for judgment, because no gate can tell a real test that matters from a real test that does not. Read before you write keeps the agent honest about the facts. Staying honest about what is worth asserting is still your job.
Or: subscribe to the newsletter for more posts on building this stuff.