Code comments written for the robot, not the human¶

Most code comments are addressed to a human who may never read them. I write a different kind of comment now: a small, typed, machine-parsed record addressed to the AI agent that edits the file next. It does not explain the code in prose. It tells tooling who wrote each fact, how much to trust it, what would break if the code is wrong, and -- crucially -- whether any of that is still true.

That sounds like over-engineering until you watch an autonomous agent edit a file that filters hidden information in a game, with no test pinning the part that must never leak. A human reviewer would feel the hair on their neck stand up. The agent feels nothing. So I gave the codebase a way to feel it for them: an annotation surface that tooling can lint, walk, and score. This post is about the source-annotation half of that system. The test taxonomy and the log-aligned UI-state dump are the other half -- I cover those in the companion post, The test that knows what it is, which is the test-and-logging side of the same machine.

The convention itself is specified in the overview docs; this is the field report on why its shape is what it is.

AI-first, not human-first¶

A conventional comment rots silently because nothing mechanically depends on it being current. A codemap: block is the opposite. Here is one, lifted from the convention doc, sitting above a view-filter function:

```python

codemap:¶

auto: engine state-machine¶

ai: hidden-info view-filter¶

human: authoritative-state¶

risk: hidden-info¶

affinity: high¶

maturity: active¶

hash: sha256:2b5a7f0c9e4d31aa¶

def player_view(state, player_id): ...

The three lead lines are provenance tiers, and they encode who may author each fact and how far to trust it:

auto: -- tool-owned facts the annotator derives mechanically: imports, path ownership, framework use, call-graph reachability. An agent never hand-edits these. The annotator regenerates them and a lint checks them.
ai: -- semantic judgment an agent can supply but tooling cannot derive reliably: the role a unit plays, the invariant it protects.
human: -- intent, policy, or domain judgment that should come from a person, or from an agent following explicit ticket context.

That split is the whole point. When an agent edits this file, it knows it is allowed to revise the ai: judgment but must leave auto: to the annotator and must not invent a human: policy. The trust boundary is in the data, not in a style guide nobody reads.

Two things keep the surface honest. First, the vocabulary is closed: every token an annotation may use lives in the controlled vocabulary, and tools reject anything not listed. You cannot coin risk: kinda-scary; you pick from a fixed set. Closed vocabulary is what makes the surface machine-consumable instead of free text that drifts into a thousand near-synonyms. (The capability tokens are deliberately shared with the test taxonomy's touches: tokens, so source annotations, tests, dependency selection, and risk scoring all speak one subsystem language.)

Second, the hash: line is a currency stamp. It covers the annotated unit, so when the code changes and the judgment is not re-reviewed, the stamp goes stale and that staleness is machine-detectable. This is the part conventional comments can never have: a mechanical way to know a comment is lying.

The enforcement closes the loop. A CLAUDE.md hard rule -- "Annotations are part of the code -- update both in the same edit" -- requires any edit to an annotated unit to re-stamp its judgment fields in the same diff, and the lint-annotations gate validates the vocabulary and the hash currency. Stale judgment is a hard error, not a code-review nit. The comment cannot quietly drift away from the code it describes, because the build stops you.

Granularity is a dial, not a ceiling¶

The obvious objection: that hash covers a whole file. If I touch one line, the stamp for everything in the file goes stale. Isn't that crude?

It is crude on purpose, and the crudeness is a dial I have chosen not to turn up yet. The hash covers the smallest stable hashable unit, and today that unit is the file: compute_unit_hash in the codemap subsystem strips the inline codemap: blocks and hashes the remaining file text. One block per file, which is the granularity an editing agent can actually keep current without drowning in annotation churn.

The deeper levels are already wired. the codemap subsystem parses each file with Python's ast and already walks ast.ClassDef, ast.FunctionDef, and ast.AsyncFunctionDef while deriving capabilities -- so the structural information needed to hash a single class or a single function is already in hand. The designed progression is file -> class -> function. Depth deepens one notch only when the payoff (pinning a judgment to one risky method instead of a 400-line module) justifies the extra annotation cost.

This is the YAGNI stance applied to metadata: start at the coarsest unit that carries real signal, and pay for finer resolution only where finer resolution earns its keep. The axis is built in; it simply has not been switched on because nothing has needed it yet. That is a very different thing from a limitation.

There is one place where I did turn the dial up: the computed maturity signal. The file hash is still file-granularity, but codemap-maturity now separates raw edit volatility from public interface stability. Raw commit count stays useful as a "this file is hot" signal. It feeds change-risk. The maturity label asks a narrower question: did the exported surface move? A body-only refactor should not make a stable contract look experimental.

For Python, the tool parses top-level exported signatures with the stdlib ast module. For JS and JSX, it uses a deliberately small regex heuristic over export statements rather than adding a parser dependency. That is not a claim of perfect JavaScript understanding. It is a targeted upgrade because the previous raw-churn maturity signal was demoting files for internal refactors, which was the wrong kind of crudeness.

Three tiers of dependency graph¶

Here is the part most projects do not build, and the reason the annotations pay off mechanically rather than aesthetically.

Almost every "what does this change affect" tool stops at static imports. Imports are real, but they are blind to half of how a system actually wires together. So the codemap subsystem builds a three-tier graph, and every edge carries its tier as a literal so a consumer can filter by confidence:

Tier A -- static import edges. What the imports literally say. deps.py parses each Python file with ast and resolves import / from ... import to in-repo modules, and parses JS import/require specifiers to relative paths. Highest confidence: the interpreter would agree.
Tier B -- protocol/convention edges. What a convention implies but no import expresses. deps.py scans client WebSocket send sites for a message_type, then links each one to the matching Room.handle_<type> handler on the server, cross-checking the pair against the documented client message list. A client send and its server handler never import each other -- Tier A is structurally blind to this edge -- yet a change to one can break the other. A missing handler or a missing protocol entry surfaces as a graph warning rather than a silent gap.
Tier C -- empirical/execution edges. What actually happened in a real session. the codemap subsystem drives the live server (a real dev-up), runs the multi-device scenario corpus against it, harvests backend logs and Playwright diagnostic timelines, and writes the codemap subsystem. Cross-device ordering is anchored on sequence and causation ids, not wall-clock, so the captured edges are the ones that only appear when the system actually runs.

The progression is static -> convention -> execution: provable from source, implied by a documented convention, observed from a run. Each tier sees edges the previous one cannot. And this graph is not a diagram for humans to admire -- the blast-radius queries that feed risk scoring walk these edges to find everything reachable from a changed file. The dependency graph is infrastructure, not documentation.

Risk is not complexity¶

The payoff is that the codemap scorer turns the annotation fields, the test coverage marks, and that three-tier graph into two distinct numbers for a proposed change -- and the distinction is the whole insight.

Complexity answers "how entangled, how much change." It sums the capability weights of each changed unit, scales by the unit's affinity multiplier (high/medium/low map to 1.35 / 1.0 / 0.75), and multiplies by a blast factor that grows with how many edges point into the unit -- capped at 2.5. Widely-depended-on code scores as more change than a leaf module with identical annotations.

Risk answers a different question: "how dangerous is it to get this wrong." Risk is computed per capability and is gated on coverage. For each capability a change touches that is not covered by a satisfying test, risk adds a term shaped like:

```text severity[cap] * maturity_risk_multiplier * expected_coverage_multiplier

On top of that, risk accumulates bonuses: a stale annotation, a missing annotation, a pins-bug mark, a raw-churn bonus per recent commit. This is the volatility axis on purpose. Recently edited code is statistically riskier to change even when its public interface is stable. In port mode, a changed unit with no behavior-preservation coverage takes the port_missing_characterization bonus -- 12.0, the single biggest flag in the table, because porting code with no characterization test is the riskiest move the system knows how to name.

That raw-churn risk bonus is separate from the maturity label. Maturity now tracks interface stability: exported Python signatures and JS/JSX exports changing over the history window. A file can therefore be high-volatility for review routing and still be stable if the contract held. Conversely, a quieter file whose exported surface keeps changing should not get stable-credit just because the commit count is low.

Two consequences are counterintuitive and exactly right:

Stable, high-blast code scores as more dangerous to change, not less. The maturity multiplier already ranks stable (1.35) above active (1.0) above experimental (0.75): mature code is load-bearing, so breaking it costs more. A stable unit whose incoming blast clears the high-blast threshold (4.0) takes a further bump. "It is stable, so it is safe to touch" is precisely backwards.
Hidden information without a covering test gets flagged. A hidden-info capability counts as covered only by a test that asserts negatively -- that a role which must not see private data does not -- and that is an invariant check. Absent that, the high hidden-info severity (8.0) lands in the risk total. This catches the exact leak class that role-based view filtering exists to prevent, before an autonomous edit can ship it.

The combined complexity + risk total then routes. It maps through thresholds in the codemap subsystem to a review tier (medium at 12.0, high at 24.0, max at 48.0) and decides whether a mandatory human_review gate fires (at 24.0). A bigger, riskier change automatically draws a stronger reviewer and, past the human-review line, blocks an autonomous merge until a person signs off.

That routing is why three different audiences want this number:

AI agents get scrutiny routed to where it is earned. A high score pulls in a stronger review tier and can block an autonomous merge -- and the hidden-info-without-test flag stops a whole leak class before it ships.
Developers see the dangerous edits before making them. Under-tested risky code is visibly expensive, which nudges the test and the annotation to land with the change instead of never.
Management gets a deterministic, auditable, portfolio-level risk signal instead of gut feel. The weights are policy in a JSON file, so the scoring can be inspected, debated, and tuned -- not relitigated in every reviewer's head.

How this compares to the prior art¶

None of the individual ideas here are unique to this repo, and I do not want the post to read as if they were. After shipping the system I did the prior-art search I had skipped, and the field has converged on most of these moves independently -- with real, shipping tools, not just papers.

code-review-graph builds a dependency graph and routes review context by blast radius -- the same "what does this change actually reach" instinct behind the three-tier graph and the blast-factor term in the complexity score.
TDAD (arXiv 2603.17973) puts tests in the same graph as the code and selects them by a confidence weight. That is the move behind gating risk on coverage and treating an uncovered capability as a risk term rather than a separate lint.
The AI-Annotations @! standard formalizes machine-addressed, agent-readable annotations carried in source -- the same premise as a codemap: block written for the agent that edits the file next, not the human who might read it.
The stale-comment patent (US8607193B2) describes detecting when a comment no longer matches the code it annotates -- the problem the hash: currency stamp solves, named in prior art years before I reached for it.

That convergence is reassuring rather than deflating: when several independent efforts land on tests-in-one-graph and impact-routed review, the shape is probably right, and saying so is more honest than implying the ideas are sui generis. What I have not found combined in the cited tools is the enforcement layer -- provenance tiers (auto/ai/human) that put the trust boundary in the data, a hash-currency lint (lint-annotations) that makes a stale judgment a build error instead of a review nit, and the Tier B message_type -> Room.handle_* protocol edges that no import expresses and that static-import graphs are structurally blind to. The ideas converged; the part that is this project's own is making them mandatory and machine-checked.

What is primitive today¶

I want to be honest about the edges. The hash unit is the whole file; class- and function-level hashing is wired but not switched on. Tier C is only as good as the scenario corpus that drives it, and a path no scenario exercises has no execution edges at all -- it falls back to Tiers A and B. The weights in the codemap subsystem are hand-tuned starting points, not a fitted model; they encode my current judgment about what is dangerous, and they will move as the catalog of shipped changes grows. None of that is hidden -- it is all in the convention doc -- but a reader deciding whether to build something similar should know the system earns its keep at file granularity and gets sharper from there, rather than arriving fully resolved.

What this means if you are building agent infrastructure¶

If you run autonomous agents against your codebase, you have probably reached for two things: a linter to catch syntax-level mistakes, and human review to catch judgment-level ones. The gap between them is enormous, and it is exactly where an agent does the most damage -- a change that compiles, passes the tests that exist, and quietly breaks an invariant no test names.

Annotations addressed to the agent, a dependency graph that knows about convention and execution edges and not just imports, and a risk number distinct from complexity are one way to instrument that gap. The annotation surface is small enough that an agent can keep it current; the scoring is deterministic enough to audit. The hard part was not the code. It was deciding that a comment should be a typed fact the build can check, not a sentence a human might read.

The other half of this system -- the test taxonomy that lets a test declare which capability it protects, and the UI-state dump aligned to the structured logs so a regression can be replayed from a timeline -- is the subject of the companion post, The test that knows what it is. That is the test-and-logging side of the same machine; this one was the source-annotation side.

Update: derive, don't stamp

The follow-up post Derive It, Don't Stamp It names the principle this annotation system was reaching for: derive routing metadata from current artifacts instead of hand-stamping labels that drift. The canonical scoring rule now lives in Change Risk Scoring.

Related work

I did eventually run that systematic prior-art search, and it has its own chapter: What the literature got to first (and the parts we made mandatory). The short version is the one the section above already tells: provenance-tiered metadata, dependency-graph blast radius, and risk-vs-complexity scoring each have deep individual literatures, and the field has converged on tests-in-one-graph and impact-routed review independently. The part I am willing to defend as this project's own is the enforcement layer -- provenance tiers that put the trust boundary in the data, a hash-currency lint that makes a stale judgment a build error, and the Tier B message_type -> Room.handle_* protocol edges that static-import graphs are structurally blind to.

Or: subscribe to the newsletter for more posts on building this stuff.