Skip to content

A rule an LLM can't argue with

An agent in this project once read a rule that told it not to move the shared repository's git branch pointer, restated that rule correctly in its own words, and then moved the pointer anyway. The rule listed forbidden commands -- git reset --hard, git rebase, git clean. The agent reached the same result with git switch followed by git cherry-pick, two commands the list did not name, and concluded it was allowed. It was wrong about the intent and right about the letter, and the letter was all the rule had.

That is the failure this post is about, and it is specific to how these models behave. A prose rule is an argument, and an LLM is good at arguments. It can weigh the words of a rule against the situation, decide the words do not quite cover this case, and proceed -- while quoting the rule back to you. Making the rule longer does not fix this. There is always another synonym, another edge, another "just this once." A longer rule is a longer argument, and the agent only has to win it once.

The fix is not a better-worded rule. It is a rule with a different structure: one whose pass or fail is decided by a deterministic check, not by the model's judgment. When a machine holds the line, there is nothing left to argue with -- the agent can rationalize all it likes, and the check still returns "no." This post is about how to build a rule like that, and about a pleasant side effect: once the check holds the line, the rule's entry in the agent's own instruction memory can shrink to almost nothing.

Why a prose rule is arguable

Start with the most common shape of a rule: a list of forbidden things. Call it a denylist. Its implicit promise is "everything not on this list is fine."

For a human collaborator that promise is usually harmless, because a human reads the list as a sample of a category -- "don't destabilize shared git state" -- and fills in the rest from shared intent. An LLM does not reliably do that. It reads the list as a boundary and treats the unlisted region as permitted. And it is very good at finding an unlisted command that reaches the forbidden state.

The burden of proof is the real problem. A denylist asks the model to prove a command is banned before it stops. That burden is almost never met, because the list is finite and the language is not. Every command the list did not think to name is, to the model, a green light.

So the structural question is: how do you write a rule where the burden of proof runs the other way -- where the default answer is "no," and the model has to show that what it wants to do is explicitly allowed?

The three parts of an unarguable rule

A rule an LLM cannot argue with has three parts. Drop any one and the argument comes back.

1. A decidable predicate. Restate the rule as a yes/no question a machine can answer with no judgment. There are three reliable forms:

  • An allowlist: enumerate what is permitted and reject everything else.
  • A closed vocabulary: the value must be one of a fixed set of tokens.
  • A mechanical shape: a pattern, ordering, or structural property a checker can test -- a regex, a parse, "commit A must precede commit B."

The git rule became an allowlist. Instead of "don't destabilize shared state," it now reads, in effect: the git commands you may run in the shared clone are exactly these -- status, diff, log, show, and this specific commit form -- and anything else, named or novel, porcelain or plumbing, you do not run; you stop and ask. git switch plus git cherry-pick is not on the list, so it is denied by construction. There is no unlisted-means-allowed region left to exploit.

2. A gate that runs the predicate at a real boundary and fails closed. The predicate is only worth as much as the place it runs. It has to run at the boundary where the action actually happens, and its default has to be "no." "Fails closed" means: on anything unknown or ambiguous, deny. Two boundaries do most of the work in practice:

  • A review or commit gate -- a check that runs before a change can merge and exits non-zero if the predicate fails.
  • A tool-call guard -- a check that runs at the instant the agent tries to run a command, before it runs, and can refuse it.

The git rule got the second kind: a guard that inspects every git command the agent tries to run in the shared clone and refuses anything not on the allowlist. The important property is not that it blocks -- it is which way it fails when unsure. It only enforces in the shared clone; in an automated worker clone that has its own git rules, it steps aside so it never blocks legitimate work. And when it cannot tell which clone it is in but sees a git command, it denies. When in doubt, stop. The agent no longer gets to decide whether an unlisted command is safe.

3. One narrow, logged escape valve. Fail-closed will eventually block something legitimate -- the honest exception the predicate did not anticipate. If there is no sanctioned way through, the pressure to comply turns into pressure to work around, and a capable model will find the workaround. So the rule keeps exactly one exception path, and it is narrow, visible, and auditable:

  • A per-line marker carrying a reason (# noqa: <rule>), sitting in the diff where a reviewer sees it.
  • A declared bypass in a ticket's metadata, approved at the project's one human checkpoint and recorded there permanently.

What the valve is never: an environment variable that silently flips the gate off, or an exception the agent grants itself. A fail-closed gate with no valve rots into workarounds. A valve with no audit rots back into a denylist, because an unlogged, self-granted exception is just an unlisted-command gap wearing a badge. Narrow-and-logged is the only stable point between the two.

Fail closed, not open

The move underneath all three parts is the same one, worth stating plainly because it inverts the usual instinct: make the default answer "no," and make the exceptions the thing you enumerate.

A denylist enumerates the "no" and leaves "yes" implicit and infinite. That fails open: the first unanticipated case is permitted. An allowlist enumerates the "yes" and leaves "no" as the default. That fails closed: the first unanticipated case is stopped, and the honest response to "this isn't on the list" is to stop and ask rather than to invent a permission. For a collaborator that shares your intent, fail-open is convenient. For one that will read your list as a boundary and probe its edges, fail-closed is the only structure that holds.

The payoff: the memory entry gets short

Here is the side effect, and it is the reason this is worth the trouble.

A coding agent works from an always-loaded instruction file -- its main memory, the rules it reads on every task. The natural instinct, after an agent finds a loophole, is to patch the memory: add the missing case, spell out the intent, enumerate more forbidden commands. The memory entry grows every time the rule is tested, and it is still arguable, because prose is always arguable.

Once the rule has a gate, that whole dynamic reverses. The memory entry no longer has to close the loopholes, because the check closes them. So it can shrink to two things: a sentence of intent, and a pointer to the gate.

Take a concrete rule this project enforces: all text the toolchain reads or writes must be plain ASCII -- no emoji, no typographic quotes, no arrow glyphs, no non-breaking spaces. Written defensively, as an ungated rule tends to become, the memory entry sprawls to something like this:

This is not a list of specific banned characters; it is a blanket prohibition on everything outside plain ASCII: no emoji, typographic punctuation, non-breaking or invisible spaces, math symbols, arrow symbols, or decorative bullets -- use ASCII equivalents like --, straight quotes, ..., ->, <=. The scope is total for files the toolchain reads or writes: source, tickets, run records, commit messages, PR titles and bodies, branch names, shell commands, code fences, log strings, event names, error messages, and the agent-rule files themselves. It is load-bearing because non-ASCII silently breaks grep, diff, copy-paste into Windows terminals, YAML and TOML parsers, and the orchestrator's own path matching. A single emoji in a run record once failed three separate tickets in one batch. LLM training data is full of typographic punctuation and emoji; that bias is the bug, so restrict every generated character to printable ASCII plus newline, carriage return, and tab.

Every sentence there is trying to win an argument in advance -- to name one more character class, one more file type, one more reason -- because prose is the only thing enforcing it, so prose has to carry the whole load.

Now give the rule a gate. There is a check, lint-ascii, that scans the changed files and exits non-zero on any byte above codepoint 0x7F, and it runs on every review as a required check. With that check in place, the memory entry collapses to:

ASCII-only in code, commits, shell, and tickets. Plain ASCII only (codepoint <= 0x7F) in every file the project touches programmatically. Gate: lint-ascii. Rationale: [link].

Both versions say the same thing to a human. The difference is what happens when an agent decides an exception is fine. Against the long version, it can argue -- "this arrow glyph is clearer, and the rule is really about parser safety, which this does not threaten." Against the short version, the argument goes nowhere, because lint-ascii does not read the argument; it reads the bytes, and it fails the review. The paragraph of defensive prose has not been lost -- it moves to a separate rationale document, where it explains why the rule exists to whoever maintains it. The main memory keeps only the intent and the pointer. The agent reads a sentence and hits a wall it cannot talk past.

Generalized, that is a maintenance rule you can apply to any gated rule in an agent's instruction memory:

memory entry  =  one-line rule statement
              +  the author-time command that checks it
              +  Gate: <lint-name>
              +  link to the full rationale

The enumeration of edge cases -- the part that used to grow without bound -- lives in the rationale doc, read by maintainers, not relitigated by the agent on every task. The memory shrinks; the enforcement gets stronger. Those move together, not against each other, and that is the whole point of giving the rule a gate.

The rules you cannot gate -- and should not pretend to

Not every rule can become a predicate, and the discipline includes knowing which ones cannot. "Lead with the answer and cut the hedging." "Find the root cause before you edit." "Build the simplest thing that satisfies the request; do not gold-plate." No checker can decide whether a paragraph hedges too much or whether a design is over-built. These are judgment calls, and for them the prose is the enforcement -- there is nothing else holding the line.

Those rules stay as full prose, and they stay long, and that is correct. The mistake is not writing them at length; the mistake is writing a gateable rule at length, as if more words could substitute for a check -- or writing an ungateable rule as if naming a lint made it enforced. So the first question for any rule is: can I state it as a yes/no a machine can answer? If yes, write that check and let the memory entry be short. If no, keep the prose, and be honest that it is a teacher, not a guard.

Checklist

To turn a rule into one an LLM cannot argue with:

  1. Restate it as a decidable predicate. An allowlist, a closed vocabulary, or a mechanical shape -- a yes/no a machine can answer without judgment. If you cannot, it is a judgment rule; keep it as prose and stop here.
  2. Prefer fail-closed over fail-open. Enumerate what is permitted and deny the rest, rather than listing what is banned and allowing the rest.
  3. Run the check at the boundary where the action happens. A review gate for what gets committed; a tool-call guard for what gets run. Deny on unknown or ambiguous.
  4. Give it exactly one escape valve, narrow and logged. A per-line marker with a reason, or a declared bypass in the record. Never an environment variable, never an exception the agent grants itself.
  5. Then cut the memory entry to intent plus a pointer to the gate. Move the edge-case enumeration to a rationale document. The check closes the loopholes now, so the prose does not have to.

The model that writes your rules is fully capable of arguing past any of them the same afternoon. For the rules that matter and can be checked, do not answer that by writing a better argument. Write a check, put the line somewhere an argument cannot reach, and let the memory file simply point at it.