Draft it rough, then try to kill it and build it

This post is about a process, not a product. The product happened to be a long design document -- a from-scratch plan for a continuously running orchestration engine that authors and runs coding tickets against a repository. About sixteen hundred lines: a daemon, a scheduler, gates, a journal format, a failure-recovery path, a build order. You do not need to know any of that. What is worth stealing is not what the plan says; it is how the plan went from a plausible-looking draft to something sound, and specifically the sequence of moves that got it there when the obvious move -- "review this for me" -- did not.

The sequence was: draft something concrete but flawed, then try to kill it, then try to build it, then compress what survived. Reading a document adversarially finds what is wrong with it. Trying to execute it finds what is missing from it. Each move has a point where it stops paying, and knowing that point is half the skill. This post inlines the mechanics of each, so there is nothing to go read afterward.

The draft has to be concrete before any of this works

The plan was assembled, not written in one pass. Each major component started as its own exploratory session -- one long conversation about the scheduler, another about failure recovery, another about the gate model, the journal, the provider routing. Each went deep on one piece in isolation and produced good local reasoning. Then a synthesis pass stitched the pieces into a single document and folded in lessons from an earlier version of the system.

That method has a signature failure mode, and naming it explains everything the reviews later had to do. The seams between the sessions are where the defects live. Two locally sound decisions, made in two different sessions, can quietly contradict when read together. A contract named in the session that produced it gets silently assumed by three other sections that never restate it. A term drifts meaning from one section to the next. None of this is visible inside any single section, because inside a section everything is consistent with itself.

Here is the part that matters as process: you want that draft anyway. You cannot adversarially review a blank page or a vague intention. There is no line to cite, nothing for an implementer to try to build. The reviews that did the real work are parasitic on a concrete artifact -- they need something that has committed to specific decisions, so that the wrong and missing ones are locatable. The vibe-and-synthesize method produced exactly that: a document confident enough to be wrong in specific, addressable places. A rough draft you can measure beats a careful draft you cannot, and trying to perfect the plan in one up-front pass is the worse move, because you would be polishing something you have no way to test. Make the concrete thing first. Its job is to be wrong in a way you can find.

Generic review polishes, and it taper fast

The first thing I tried was the obvious thing: hand the whole document to a fresh model and ask it to review it. Find problems. Anything wrong here. I did this more than once.

It was not useless. It produced two rounds of edits -- a first pass of "do these now" fixes, then a second pass of "should-do and nice-to-have" refinements. But nothing structural moved, and the two rounds already show the shape of diminishing returns: the second round's own label is the tell. You do not name a pass "nice-to-have" while it is still turning up must-fix defects. Round one found the reachable real problems; round two was mostly scraping the margins and re-confirming. The taper from "do this now" to "you could consider" is not the plan getting good -- it is the technique getting exhausted.

The reason generic review saturates so quickly is that it rewards agreement. The helpful move, for a model handed a mostly-coherent document, is to confirm it is mostly coherent and suggest improvements at the edge. It will not readily stake out the claim that a central decision is unsound, because nothing asked it to take that risk, and the document reads -- section by section -- as though it knows what it is doing. "Review this" also measures nothing: it has no standard for specified enough to build, so it returns opinions, and opinions about a plausible document trend toward approval. Re-running it does not help; the first run already removed the defects that framing can see. To move the plan I had to stop asking for review and start designing the reviewer's incentive.

Move one: try to kill it

The first designed review had one job: decide whether to kill this plan before anyone builds it. The framing is the mechanism. The reviewer is not helping me improve a document; it is deciding whether the document deserves to be built from at all, and it is told to assume the plan is flawed until the text proves otherwise. That one reframing -- from "improve this" to "should this die" -- flips the default from agreement to scrutiny.

The prompt then pinned the output so scrutiny could not dissolve into commentary. The full set of rules it ran under:

• Lead with a verdict, no preamble: viable / viable-with-changes / not-viable-as-written, plus a confidence level and what would raise it. A forced verdict forces prioritization -- you cannot render a kill/keep call without deciding which problems actually bear on it.
• Rank findings by how likely they are to sink the project, in four buckets: soundness failures (logic that does not hold, decisions that contradict, steps that cannot be built in the stated order); gaps (what must be specified to execute and is not, citing the section that should have contained it); unstated assumptions (what has to be true, and which of those are unproven); and the single weakest link with its blast radius.
• Every finding cites a line number or heading. No finding without a location. A defect you cannot point at is one you are probably inventing.
• No praise unless it is load-bearing to the decision. Strengths do not change a kill/keep call, and every line spent on them dilutes the ones that do.
• Separate "this is wrong" (fact) from "I would do it differently" (opinion), and label each. Taste and genuine defects look identical in ordinary review; the label keeps real problems from drowning in preference.
• Say "no issue found" when a thing is genuinely fine, and "unassessable -- missing X" when a section is too vague to evaluate. Do not invent problems to seem thorough, and do not charitably guess past a gap.

That last clause is the one doing quiet work. "Unassessable -- missing X" is the review refusing to do the author's job. A normal reader hitting an underspecified passage fills the gap from their own understanding and reads on. This reviewer is told to stop and name the gap instead.

The output was structural in a way the generic rounds had not been. It forced the journal file layout to be pinned down instead of gestured at. It killed a cross-platform hedge that was adding complexity to defend against a portability problem the project did not actually have -- which let the first version pin to one platform and get simpler. It turned an unstated assumption into a mechanism: the plan assumed its gates would stay useful, and the pressure on that assumption is why the plan grew a way to mechanically measure whether each gate is still catching anything, rather than trusting that it is.

The kill-review has a ceiling, and seeing it is what motivates the next move. It is still adversarial reading. A careful reader -- human or model -- can talk themselves into believing a section is clear when it is not, because they are silently supplying the missing pieces from their own context. Reading tests coherence. It does not test whether a stranger with no context could act on the document. For that you have to make someone act on it.

Move two: try to build it

The plan claimed, in its own opening lines, to be self-contained: an implementer needs no other document. That is a claim about what happens when someone writes code from it, and no amount of reading can falsify it, because a reader brings context to fill any gap. Writing is the falsifier. So the second review made fresh agents role-play the implementer, try to build from the document alone, and it measured every place they had to make something up.

The setup is built to produce a clean signal:

• Three implementer agents, identical prompt, fully isolated. Each gets the plan and nothing else -- no repository, no history, no awareness the others exist. The isolation and identical prompts are what make the central statistic mean something.
• Each produces a repo skeleton, interface stubs (signatures only, no bodies -- evidence of a real read), and the actual deliverable: an invented-decisions ledger.
• The ledger has one entry per choice the document did not make for them: the decision in a sentence; the section that should have determined it; what they chose and why; and a class -- missing (the document is silent), ambiguous (two readings both defensible), contradictory (two passages disagree), or deliberate-freedom (plausibly left to the implementer on purpose). That last class is calibration -- naming the on-purpose freedoms shows the agent could tell the difference.
• Each also produces a BLOCKED list: anything it could not even invent past without risking a stated contract. An empty BLOCKED list is a valid, good answer.

The agents were told to build as specified -- not to evaluate or improve, and to follow the document even where they disagreed. This is the opposite instruction from the kill-review, on purpose. The kill-review's job was to judge; the builder's job is to comply, so that every place compliance is impossible becomes a measured data point instead of an argument.

The mechanism that makes this more than three opinions is convergence. A decision one isolated agent invents might be that agent's taste. A decision that two or three isolated agents, unable to talk and given the same prompt, each independently had to invent is not taste -- it is the document failing to determine something it needed to. Three strangers reaching for the same missing spec is objective in a way one reviewer's "this seems underspecified" never is. Isolation keeps the count clean; identical prompts keep it comparable; replication turns an impression into a number.

A fourth agent -- the synthesizer -- reads the three ledgers against the document and does four things. It clusters entries describing the same gap (three agents word it three ways). It ranks: BLOCKED first, then gaps hit by three of three agents, then two of three; within a tier, contradictory ahead of ambiguous ahead of missing. It reports what each implementer chose, because three different inventions for one gap is worse than three matching ones. And it verifies each cluster against the document: if the answer is actually there and all three simply failed to find it, that is not a content gap but a findability problem -- the answer exists, buried where no implementer looks. The two need different fixes: write the missing answer, versus move or signpost the present one. Conflating them wastes effort, so the synthesizer keeps them apart.

The skeletons and stubs are then thrown away. They were measurement instruments, not the start of an implementation. The only thing that survives is the ranked ledger -- the measurement itself.

The yield: the synthesis returned thirty ranked gap clusters and five findability problems, and those thirty-five findings became edits. Thirty-five places where a stranger building the system either had to invent a decision the document owed them or could not find one the document had made. Not one had surfaced in any amount of reading, because reading cannot feel the absence of a decision you are unconsciously supplying yourself.

Move three: compress what survived, without technical loss

After the gaps were filled, one more refactor mattered, and it is a different kind from the others: it removed nothing true and added nothing new. The plan is itself an input to context-limited agents -- the eventual implementer reads it, and so does every review agent above -- so its length has a real cost measured in tokens and in how much prose a defect can hide behind. So the plan got a compression pass: cut the rhetoric, the hedging, and the restatement, while preserving every contract, decision, and vocabulary term unchanged in meaning.

The discipline is the whole point. This is compression for the context budget without technical loss -- a shorter document that a machine reads to the identical conclusion. It is worth separating from the gap-filling because it is a different success test: gap-filling changes what the document determines, compression changes only how densely it says it. A leaner plan is strictly better when the plan is machine-read, and it has a second-order payoff: with the padding gone, the next review is sharper, because there is less prose for a gap to hide behind.

Knowing when a move is spent

Each of these techniques pays well and then stops, and the honest part of the process is recognizing the stop rather than re-running a saturated prompt hoping for more.

Each has an end-state that signals exhaustion. Generic review tapers from "must-fix" to "nice-to-have" -- that slide is the signal, not a sign the plan is finished. The kill-review is spent when its verdict stabilizes to "viable, high confidence" and its remaining findings are all opinion-class "I would do it differently." The dry-run is spent when the BLOCKED lists come back empty and a fresh run of three isolated builders stops producing new clusters -- convergence dries up, which is exactly the same statistic that made the gaps visible in the first place, now reading zero.

The trap to avoid: a spent technique is not a correct plan. A kill-review with nothing left to say means there are no more readable defects -- not zero defects. A dry-run that returns empty means the document is self-contained enough to build from -- not that the design is right. That is why the moves are a sequence and not a loop over one prompt. Each measures a different property -- coherence, then completeness, then density -- and saturates on its own. You run a move until its yield drops to noise, then switch to one that measures something the last could not see, rather than squeezing a dry hole.

And the whole document-side sequence still bottoms out at a property no review of prose can establish: whether the built system behaves. In this plan that question is deliberately deferred to the first real build phase, which carries an explicit go/no-go gate against a seeded set of planted defects before the system is trusted to run on its own. The reviews got the document to the point where building it was a well-defined act. Whether the act succeeds is a later measurement, taken against reality instead of text.

The process, stripped of my specifics

1. Draft something concrete first, even seam-riddled. The reviews are parasitic on a committed artifact; you cannot kill-review or dry-run a vague intention. A rough draft you can measure beats a careful one you cannot. Do not perfect it up front -- its job is to be wrong in locatable ways.
2. Do not ask for review; design the reviewer's incentive. "Any problems?" rewards agreement and returns polish that tapers within two rounds. Decide what you want found and write a prompt whose success condition is finding it.
3. To find unsoundness, make the reviewer try to kill it. Force a viable / not-viable verdict up front. Rank by blast radius. Require a line citation per finding. Split "this is wrong" from "I would do it differently." Allow "no issue found" and "unassessable -- missing X" so it neither invents problems nor guesses past real gaps.
4. To find missing specification, make fresh isolated agents try to build it. Several agents, identical prompt, no shared context, each logging every decision the document forced them to invent. Treat cross-agent convergence as the signal, and separate genuine silence from a buried-but-present answer -- a findability defect -- because they have different fixes. Keep the skeletons; discard them. Only the ledger feeds back.
5. Then compress for context without technical loss. Once the document is complete, cut rhetoric and restatement while preserving every contract verbatim in meaning. A machine reads a shorter document to the same conclusion, and the leaner text makes the next review sharper.
6. Run each move until its yield drops to noise, then switch -- do not re-run. The tapers are legible (nice-to-haves, a stable verdict, empty BLOCKED lists). A spent technique means no more defects of that kind, not a correct plan. Switch to the move that measures the next property, and remember the last property -- does the built thing work -- is not a document review at all.

The honest edges

None of this proves the system works. It proves things about the document.

The kill-review can only judge what it can read; a decision that is coherent, cited, and clearly stated can still be wrong about the world, and a reader who accepts the plan's own framing will not catch a well-argued bad idea.

The dry-run has sharper limits. Convergence detects gaps, not bad decisions -- if all three builders happily accept a specified decision that is wrong, they invent nothing and the ledger stays silent, because there was nothing to invent. Three agents is a small sample, so a real gap only one builder happened to hit still exists; those go in a low-confidence tail rather than the trash, because one-of-three is weak evidence, not zero.

And the "draft first" framing has a precondition it is easy to miss: the draft has to actually commit. A draft that is itself vague starves both reviews -- the kill-review returns "unassessable" everywhere and the builders invent past nothing because there is nothing to invent past. Do-something-first only works if the something makes decisions. The reviews are good at finding decisions that are wrong or missing. They are useless on a document that declined to decide.

That division is the point. Reading the plan tells you whether it is coherent. Trying to build it tells you whether it is complete. Compressing it makes it cheaper to read without changing either. And only running the built system tells you whether it is right -- which no review of a document, however adversarial, can stand in for.