Risk-routed review: spending human attention where it earns its keep¶

The last bottleneck in an automated build loop is not typing code. It is not running tests. It is not even spawning enough agents. Once the workflow can draft tickets, run them in disposable clones, execute gates, and hand back reviewable diffs, the scarce resource is obvious: a human who can still notice the change that is formally green and substantively wrong.

That is the uncomfortable success case. Automation does what it is supposed to do and produces more work than a person can inspect line by line. If every diff gets the same review depth, the system either slows to human speed or teaches the human to skim everything. Both outcomes waste the one resource the robots cannot replace: judgment.

So the workflow has to route risk. Not because a number can prove a change is safe, but because a number can decide where the human should spend their next minute.

The gate layer is necessary, but it is not the whole bottleneck¶

The previous orchestration post, Review surfaces and the AI-first guardrails, describes the gate layer: review preflight, surface-specific rule packs, bug-fix proof requirements, annotation currency, test marks, docs builds, and the standing agent rules that make the wrong shape fail mechanically.

That layer answers a concrete question: did this ticket satisfy the contracts that can be checked by the machine?

It does not answer the next question: if twenty tickets satisfy those contracts today, which two should a human read first?

That is a different problem. The gate layer shrinks the blast radius of bad agent work. Risk-routed review allocates the remaining human attention after the gates have done their part.

Routing point one: author-time grouping¶

The first risk route happens before any implementation clone is spawned. Tickets can carry a coarse predicted_risk: low | medium | high field: the author's pre-estimate of how dangerous the change is likely to be. It is not the precise score. It is a planning signal, written when the work is still being split. When the field is absent, group-suggest treats the ticket as medium, the neutral default: not safe enough to hide in a purely low-risk batch, and not dangerous enough to force high-risk handling by itself.

group-suggest uses that signal as a secondary pressure when proposing batches, and then enforces the clean-split case in deterministic post-processing. Theme and coupling still come first. Tickets that share ## Touches, outputs, or cross-cutting contracts stay together even if their risk labels differ. But when there is a clean split, high-risk work is split out of a mostly low- or medium-risk batch instead of being dissolved into it.

That gives the human a better review unit. Instead of one large mixed batch where risky work hides beside routine edits, the planner can concentrate the high-risk tickets into a small focused group and leave the lower-risk group for lighter review. The printed group summary also reports the concentration label and low/medium/high counts, so the reviewer can see whether a batch is high-isolated, high-mixed, or lower risk before running it. The point is not to pretend the author estimate is exact. The point is to stop throwing every kind of work into the same review bucket.

The batch-authoring side of the workflow documents this as a planning pattern in the workflow docs: use the coarse author-time signal to shape the batch, then let review-time scoring decide how much scrutiny the actual diff earned.

Routing point two: review-time scoring¶

The second route happens after the code exists. The codemap scorer reads the actual diff and emits complexity, risk, and routing data. The worked formula is in Change Risk Scoring; the important shape is that the scorer combines:

source codemap: annotations, including capabilities, affinity, maturity, risk flags, and annotation currency;
test covers: marks and evidence tags;
dependency-graph blast radius, including direct test edges;
recent churn and risk bonuses;
routing thresholds from the codemap subsystem.

The combined complexity + risk total maps to a reviewer tier and to the human_review gate. A larger or structurally riskier change can draw a stronger reviewer automatically, and past the human_review threshold it cannot merge autonomously without a person signing off.

This route is intentionally later and sharper than predicted_risk. The author estimate is a coarse planning label. The review-time score is based on the code that actually changed and the coverage shape around it.

Why concentration beats uniform review¶

Uniform review sounds fair, but in an automated workflow it becomes a trap. If every ticket gets deep human review, throughput collapses to the old bottleneck. If every ticket gets light review, the system invites exactly the failures the gates were built to expose: hidden-information leaks, routing regressions, stale annotation judgments, or tests that prove an agent's premise instead of the user's repro.

Risk concentration gives the human a smaller, more useful target. A batch full of high-risk tickets tells the reviewer what kind of attention to bring: scrutinize invariants, hidden state, routing boundaries, blast radius, and coverage claims. A low-risk batch can still pass through the normal wall of machine gates, but it does not consume the same scarce human budget by default.

The benefit compounds with the review-time router. Author-time grouping decides which bundle should be looked at first. Review-time scoring decides which specific diff inside that bundle earned a stronger reviewer or mandatory human_review. One is a planning tool; the other is a gate.

The honest limits¶

The author-time estimate is coarse by design. A ticket author can be wrong. They can underestimate a cross-cutting path, overestimate a simple docs change, or leave the field absent when the shape is unclear. That is why predicted_risk is a grouping pressure, not an override. Coupled work still stays together, and the computed review-time score remains the precise authority once the diff exists.

The review-time score has limits too. It is structural. It can see that a changed unit has hidden-info, that a test claims the capability, that a direct test edge exists, and that evidence tags are present. It cannot prove the assertion is the right assertion. A weak test with the right marks can still look stronger than it deserves, and real behavioral coverage can be invisible if the marks are missing.

That is why the human gate remains the safety valve. Risk routing does not replace judgment. It protects judgment from being smeared evenly over work that does not all deserve the same depth.

The reusable idea¶

The move here is smaller than the tooling: split risk routing into two times. At author time, use a coarse estimate to keep the planner from burying risky work in broad mixed batches. At review time, score the actual diff and route reviewer strength plus the mandatory human gate from the structure of the change.

That two-step shape is what lets automation scale without pretending humans are infinite. The robots can write, test, annotate, lint, and propose batches. The human should spend attention where it still changes the outcome.

Or: subscribe for more posts on building autonomous AI development infrastructure.