Skip to content

Battleland: the first game that trains its own opponent

Most of the games in this catalog do not train anything. The opponents you play against -- Steady Eddie, The Prodigy, Old Reliable -- run a tree search over the game's rules and pick a move. The search is real work, but it is search: the program reasons forward from the position in front of it every single turn, from scratch, using rules a human wrote. Nothing is learned and nothing is saved. Turn the program off and it forgets nothing, because it never knew anything.

Battleland is the exception. Battleland is the one game in the catalog with an end-to-end learned-model pipeline: a neural-network-shaped model that plays thousands of games against itself, keeps a record of what worked, and updates itself to play better next time. This is the "we actually trained a model" story, and if you are new to AI-assisted development, it is worth understanding in detail -- not because you will retype it, but because the shape of this pipeline is the shape almost every modern game-playing model has, all the way up to the famous ones.

This post walks the real pipeline that ships in the repo. Every file and symbol named below is something you can open and read.

What "a self-play pipeline" is made of

The AI lives under the relevant AI module, and the relevant AI module tells you the public pieces at a glance. There are four moving parts, and they form a loop.

1. A combined policy/value network (net.py, the class BattlelandPolicyValueNet). This is the model -- the thing that gets trained. It has two outputs, called "heads," and the trick that makes AlphaZero-style systems work is that one model produces both:

  • The policy head looks at a position and the list of legal moves, and returns a weight for each move -- a hunch about which moves are worth examining. It is keyed by the actual Action objects the caller passed in, so the answer is "here is how promising each of these moves looks," not a guess about moves that are not even legal.
  • The value head looks at a position and returns a single number in [-1, 1] from the active side's perspective: 1.0 means "the side to move is winning," -1.0 means "the side to move is losing," 0.0 is even. One scalar. That scalar is the model's entire opinion of who is ahead.

2. A self-play loop (selfplay.py, the function generate_self_play_game). This sits the model down to play a full game of Battleland against itself, recording every decision it made along the way.

3. A tree search that uses the model's hunches (search.py, the function search). This is Monte Carlo Tree Search -- the same family of search the other games use -- but wired so the model's policy head biases which branches get explored and the model's value head replaces the usual "play random moves to the end and see who wins" step. Instead of guessing at the end of a line by flailing randomly, the search asks the value head who is ahead.

4. A trainer (train.py, the functions train_one_iteration and train_step). After a self-play game finishes, this nudges the model's parameters so that next time it would lean a little more toward the moves that turned out well and judge positions a little more like the outcomes it actually saw.

Two more entry points round it out. inference.py (choose_action and evaluate_state, wrapped in the BattlelandAI class) is how the rest of the system asks the trained model a question -- "what move would you play here?" or "who is winning here?" -- without knowing anything about training. And strike_preview.py is a separate, smaller module of closed-form dice math (the binomial hit-count distribution for a declared strike); it is not part of the learned loop, but it shows up again when we get to the observer view.

The loop, concretely

Here is the part that surprises people the first time: nobody hand-labels anything. The model teaches itself from the only honest signal a game provides -- who won.

Read generate_self_play_game and you can watch it happen. For each decision in the game, it runs search from the current position. The search returns two things worth keeping:

  • The visit counts -- how many times the search bothered to explore each legal move. Searches spend their time on moves that look good, so the visit distribution is a refined opinion about the position, better than the raw policy hunch the search started from. Normalized, this becomes the policy target: "next time, your policy head should look more like this."
  • The chosen move, which gets applied to advance the game.

The loop records a training sample at every decision, but with one field deliberately left blank: the value target starts at 0.0. The model does not yet know whether this position was a good one to be in, because the game is not over. Only when the game ends does generate_self_play_game go back and fill in every sample's value target with the eventual result -- +1.0 for the side that won, -1.0 for the side that lost, 0.0 for a draw (see _value_target_for_player). Every position the winner passed through is retroactively labeled "this was a winning position"; every position the loser passed through, "this was a losing one."

Then train_step walks those samples and nudges the network: the value head toward the real outcomes, the policy head toward the search's visit distribution. train_one_iteration ties it in a bow -- generate one self-play game, train on it, return the improved model. Run that thousands of times and the model that started out knowing nothing has taught itself to evaluate Battleland positions, using games it generated itself, judged by an outcome no human had to annotate. That self-referential loop -- play yourself, learn from the result, play yourself again with the better model -- is the whole idea behind "self-play," and it is why this is the catalog's strongest "we trained a model" story.

Why a wargame uses vanilla AlphaZero

Here is the design decision that makes Battleland a good teaching example, and it is the kind of decision an AI-assisted developer has to learn to make: the algorithm you reach for is dictated by the game's information structure, not by how hard the game looks.

Battleland is a hex-map skirmish wargame. It looks intimidating -- hundreds of legal action-plans per turn across movement and strikes, a stochastic dice-combat layer, titans, terrain. You might assume a game that big needs the fanciest search in the toolbox. It does not, and the design doc says so directly. From the Battleland design doc, Section 10:

Battleland is fully observable ... Vanilla AlphaZero suffices; the determinization layer that RAIL_ROUTES leans on for hidden-info isn't needed here.

"Fully observable" is the load-bearing phrase. Both armies are on the board, face up. There are no hidden hands, no face-down cards, no secret legions. When the model evaluates a position, it is looking at the whole truth of the game -- exactly the situation plain AlphaZero was designed for. The hard part of AlphaZero is search and learning, and Battleland needs both; the part Battleland gets to skip is reasoning about information it cannot see, because there is none.

Contrast this with Bluff, the catalog's poster child for hidden information. In a game built on lying, the entire game is the cards you cannot see in other players' hands. A model there cannot evaluate "the position" because there is no single position -- there is a cloud of possible worlds consistent with what has been claimed so far. That is what forces the heavier machinery: Information-Set Monte Carlo Tree Search (ISMCTS), which samples possible hidden states and searches across them. Battleland needs none of that. Its design doc is blunt about it: "Hard-tier ISMCTS is overkill for Battleland's fully-observable skirmish."

So the same project ships two AIs that look superficially similar -- both are Monte Carlo Tree Search at heart -- but one runs vanilla because the board is honest and the other runs ISMCTS because the game is built on lying. For a developer learning to work this way, that contrast is the lesson: you do not pick the algorithm by the size of the game tree; you pick it by what the player is and is not allowed to know.

The value head has a second job: the observer payoff surface

A trained value head is useful for more than picking moves. It is a real-time estimate of who is winning, and that turns out to be exactly what a spectator wants.

Battleland's observer view -- the read-only role for someone watching a match they are not playing -- gets a data layer the players themselves never see. The front-end component is the private source tree, and it renders three panels:

  • Win probability. The server takes the trained value head's [-1, 1] opinion of the active side and maps it to side-by-side odds for attacker and defender. The wiring lives in the relevant server module (derive_win_probability), which calls the model's evaluate_state and, honestly, reports "unavailable" when no trained model is loaded rather than faking a number. This is the model's payoff surface made visible: the same scalar that steers self-play, shown to a spectator as a live read on the match.
  • A contested-hex heatmap. Which hexes both armies are fighting over, scored and ranked (derive_contested_hex_heatmap).
  • A dice-distribution sanity check. Cumulative actual rolls versus what the math expected -- the same closed-form expectation that strike_preview.py computes per strike -- so an observer can eyeball whether the dice have been fair.

The crucial design rule, from the design doc Section 22.2, is that all three are observer-only by privacy design. Players do not get a live win-probability readout mid-match -- the doc calls out that a dice-streak panel during play would just breed "mid-match streak-anxiety." The trained model's opinion is a spectator amenity and a post-game lens, deliberately not a coaching crutch handed to the people still playing. The same server that owns the truth of the game decides who is allowed to see the model's read of it.

The honest part: this is a tiny stub at LAN scale

Now the part a good explainer has to say out loud, because the value of this post is the pipeline shape, not a brag about scale.

The model that ships today is small. Open net.py and you will find that BattlelandPolicyValueNet is, in its own words, "deliberately a tiny deterministic linear model rather than a real ML framework." There is no PyTorch, no GPU, no million-parameter network -- the value head is a tanh over a handful of weighted features, the policy head is a lookup over action-type priors, and the whole thing is stdlib-only and fully deterministic. No frontier model is involved. No giant training run has happened. If you were hoping for "we trained a 70-billion-parameter wargame brain," this is not that, and pretending otherwise would be the kind of fabricated-stat dishonesty this project tries hard to avoid.

What matters is that the interface is the real AlphaZero interface. The module is explicit that it is "shaped so a later ticket can replace the parameter store and fitting routine with a larger checkpointed model without changing search, self-play, inference, or observer value-head callers." The self-play loop, the search wiring, the trainer, the inference entry points, and the observer's value-head read are all already in place and already correct. Swapping the tiny linear stub for a checkpointed, properly-trained network is a parameter-store change behind a stable seam -- not a rewrite. The design doc's own sizing intuition is that training to strong-amateur strength is plausible on a consumer GPU in days; that is a forward-looking estimate, not a result that has happened yet, and this post is careful to keep those two things separate.

That is the honest edge, and it is the right one for a learning developer: you do not need a data center to build the shape of a self-play training pipeline correctly. You need a clean seam between "the model" and "everything that calls the model," a self-play loop that labels its own data from real outcomes, a search that trusts the model's hunches, and the discipline to keep the model's opinion on the right side of the privacy boundary. Battleland has all of that running at LAN scale with a stub you can read in one sitting. The scale comes later; the architecture is the lesson.

If the move-grading personalities are the catalog's answer to who an opponent should be -- see AI opponents that play like people, not optimizers, where the design doc notes those personalities eventually become policy mixings on exactly this trained net -- then Battleland is the catalog's answer to a more basic question: did the opponent ever actually learn anything? Here, for once, the answer is yes, and you can watch it happen four files at a time.