Running Gemma 4 as a Commander in a strategy game
The question in this post comes from Can LLM Agents Actually Play Strategy Games by Lossfunk, and the experiment below is my attempt to answer it with a small model and a real game engine.
I connected google/gemma-4-E2B-it to the real 0 A.D. Release 28 engine and let it command a Spartan civilization against the game’s built in PetraBot. The model lost both times. The interesting part is that the model never made an illegal move, and it still lost by a factor of more than two, because the interface I gave it could not express the move that wins.
The code, the notebooks, the replays, and the videos are on the age-of-llms branch of markov-chainsaw.
How one decision works
0 A.D. ships an official reinforcement learning interface, so the engine runs headless and accepts commands over a local socket. I read the game state, flatten it to JSON, and ask Gemma for one macro action. The reply has to pass an allowlist of 13 actions, a legal action list computed from current resources and population room, and a cost check, before an adapter turns it into an engine command.
Two properties of the loop matter later. First, the engine keeps simulating between decisions, and it does so with no orders at all. Second, each prompt contains the current state and nothing else, so the model cannot see what it ordered last time or whether the order worked.
The two runs
The first run was a plumbing test. It gave the model 30 decisions with 15 engine turns between them, and it confirmed that the pipeline held together, with zero parser fallbacks across all 30 replies. It recorded no final comparison, because the scoring code did not exist yet. The honest summary of run one is that it proved the model could play legally, and it measured nothing about whether the model played well.
The second run, exp-rts-001, added the missing measurement and extended the match.
| Run one | Run two (exp-rts-001) |
|
|---|---|---|
| Model | gemma-4-E2B-it, 4-bit |
same |
| Seed | 42 | 1001 |
| Model decisions | 30 | 40 |
| Engine turns between decisions | 15 | 25 |
| Total engine turns | about 450 | 1,000 |
| Parser fallbacks | 0 | 0 |
| Final score | not recorded | Petra 626.0, Gemma 264.5 |
Both sides were still alive at turn 1,000, so the experiment fell back to a turn limit score, defined in the repository as population + 3*workers + 6*military + 8*structures + resources/100. I checked the arithmetic against the published final state and it reproduces both totals exactly.
| Metric | Gemma | Petra |
|---|---|---|
| Phase | village | town |
| Population | 18 of 20 | 69 of 70 |
| Economic units | 17 | 60 |
| Military units | 15 | 40 |
| Structures | 12 | 16 |
| Food | 30 | 410 |
| Stone and metal | 600 | 340 |
| Turn limit score | 264.5 | 626.0 |
The score is not a neutral measurement, because infantry count as both workers and military, so a large army is worth roughly nine points per unit. The bias favours Petra here, and the phase and population columns tell the same story without any weighting.
The videos
Both videos come out of the game’s own replay renderer under a virtual display, so the footage is the actual simulation rather than a reconstruction. The first is the 30 decision run.
The second is the full 1,000 turn match, which ran for 599.8 simulated seconds and was fitted into 45 seconds of video. The animation at the top of the post is the same match at lower quality.
What went wrong
Going from 450 turns to 1,000 turns did not help, and the reason is that the bottleneck was never the length of the match. Run two gave the model 33 percent more decisions spread over more than twice as much game time, so its control density actually fell from one order every 15 turns to one every 25. For 960 of the 1,000 turns, nobody was giving the Spartans orders.
The deeper problem is the action list itself.
Nothing in the 13 actions costs stone or metal. Two of the four gathering actions were therefore pure waste, and they stayed in the legal action list on every single turn, so the model was repeatedly offered moves that could not change its score. Gemma ended with 600 stone and metal it had no way to spend, and 30 food, which is what it actually needed to train anything.
The list also has no action for advancing a phase and no direct action for raising the population cap. Petra reached the town phase and a cap of 70. Gemma stayed in the village phase at a cap of 20, so it was population blocked for most of the match with 950 banked resources. The single highest value move in 0 A.D. was not in the vocabulary, and no amount of model quality can recover from an omission of that kind.
One more effect is visible in the code, and I want to flag it as an inference rather than a measured result. build_house assigns the first three workers to a foundation, and any following gather order selects workers from the same front of the same list, so a gather order issued after a build order will pull the builders off the site. Twelve structures alongside a population cap that never moved from 20 is consistent with houses that were started and never finished. Per decision logs would settle it, and the published run does not include them.
A proposal for using RTS games to test language models
An RTS match is a useful test for language model agents because the engine is the verifier. There is no reward model to fool and no judge to persuade, since the simulation either grows your economy or it does not, and the result is deterministic and replayable from a seed. The task also has the properties that agent benchmarks usually lack, because it runs for hundreds of steps, it hides part of the map, it forces a budget allocation every minute, and it includes an opponent who punishes a bad plan.
Training a model in the setting would answer a specific question. A model can state a plan in fluent language, and the open question is whether its next 40 actions match the plan it just stated. An RTS match measures plan consistency directly, because the stated reason for each action is logged next to the state that followed it, and drift between the two is visible without a human rater.
Four changes would make the setup a real experiment rather than a demonstration. First, give the model a complete action vocabulary, including phase advancement, and put the cost of each action in the prompt so the model can reason about what it can afford. Second, carry a short history of recent orders and their outcomes, so the model can notice that its last three orders changed nothing. Third, replace the fixed clock with event triggers, so the model is asked for a decision when it is attacked or population blocked instead of every 25 turns. Fourth, train against the engine score with self play across rising PetraBot difficulties, and hold out map seeds to check that the policy generalises.
The result would apply outside games. Any agent that has to spend a limited budget over a long horizon, while an environment changes underneath it, has the same shape as an RTS opening, and incident response and capacity planning are two ordinary examples. A game engine gives that class of problem a cheap and fast ground truth, which is exactly what is missing from most current agent evaluations.