I built a racing demo where identical cars compete on a closed circuit, each one with a pilot that reacts frame by frame and a team boss — an LLM — that reasons per event and radios back strategy directives. Eight attempts at training the pilot with reinforcement learning didn’t work — the track was broken before the algorithm ever had a chance. The team boss did work, and an experiment on how much it actually contributes ended up more interesting than the demo itself: seven runs, two real and partially fixable reasoning biases, an infrastructure bottleneck, and a lesson about trusting the full benchmark over the quick desk-check.
Game development has always caught my attention, Unity in particular and the whole industry that moves around it. I never got the chance to break into that industry — different paths, different contracts — but with today’s coding assistants that becomes reachable in a way it simply wasn’t a couple of years ago. I had Claude Code handle almost the entire Unity side, which isn’t my strength, and kept the design decisions and the part I actually care about for myself: what happens when you give an agent real autonomy over another piece of the system.
Because the hook was never “build a game.” The actual idea comes from Formula 1, from watching it as a kid without thinking about any of this yet: what caught my attention was never the car or the driver, it was the team radio. The strategist on the pit wall sees the whole race — standings, gaps, the timing board, the entire race plan — and sends the driver a short, specific instruction: attack now, manage the tire, defend the position. The driver sees none of that; he’s doing 300 km/h reacting to whatever’s ten meters ahead, with no room to think about the broader strategy. Two tiers, two rhythms, a narrow command channel between something that reacts and something that reasons. I connected that image to LLMs much later, thinking about how they reason with the whole context in view instead of reacting to one data point at a time: the pit-wall strategist already is, at bottom, exactly that — it sees the full picture and gives a high-level instruction, without ever touching the wheel itself.
And lately I’ve been deep in agentic communication — MCP, A2A, and generally how far the idea of agents coordinating with each other can be pushed, including agents guiding “dumb” devices (I’ve got an ESP32 experiment in mind for later). The pit-wall pattern is exactly that: a narrow command channel between something simple and something that thinks. This demo is a rehearsal of that idea with a steering wheel and six cars, before trying it with real hardware.
A pilot that never learned to drive
The initial plan was reasonable on paper: a short closed circuit — so a training episode is one lap, not the whole race — with raycasts, speed, and angle relative to the ideal racing line as observations, PPO as the algorithm, and a progress reward with a penalty for going off-track or crashing.
I ran eight full iterations, each 6 to 10 million steps, changing one variable each round: the reward shape (seven different versions), a reference heuristic I used purely for diagnosis (eight versions), the turn radius, the wall geometry — I tried zero-thickness mesh, boxes, extruded solids — the spawn point, the lateral grip model. The number I actually cared about, how much of a lap a car completed without going off-track, never moved off 10-13% across all eight.
What broke the plateau wasn’t touching the algorithm. It was the diagnostic heuristic, which on the same track and the same physics reached 82% of a lap at a sustained 21 m/s. Same environment, one method solves it reasonably well and the other gets stuck at a tenth of that — at that point it stopped making sense to keep thinking the problem was the reward.
The change that did move something was reshaping the speed reward to have a peak, not a slope. Before, more speed always scored more points — so braking for a corner was pure cost, never a benefit, and the agent learned exactly that: never brake. The reform computes a target speed from the upcoming curvature —
where is the largest heading change on the track over the next 30-55 meters — and rewards getting close to that target, not exceeding it:
With this, braking before a tight corner stops being pure cost — there’s a zone where braking is worth more than not braking. Giving the agent lookahead on top of that (the same forward-curvature observations the heuristic used in ) raised the reward’s value by 40%, but the lap percentage didn’t move: the agent still never discovered the cornering maneuver — brake, turn, accelerate, a coordinated one-to-two-second sequence — because 97% of attempts died before those two seconds. A hard exploration problem, not a badly designed reward.
All eight runs used PPO end to end; what changed between them was the reward, the agent’s perception, or the physics — never the algorithm itself. Before the last one, I tried imitation from the heuristic as a bridge — recorded demonstrations and mixed behavioral cloning with GAIL — but the gain was small, so the final run went back to clean PPO, without that scaffolding, to get a clear read on whether the algorithm alone was enough. I never got to try curriculum learning or a bigger network — they were noted down as the next step if this didn’t work, but the project got descoped before that iteration.
In parallel, a second variable was contaminating the whole experiment without my knowing: the procedurally generated tracks had broken geometry in several stretches, corners that honestly not even a human could have taken cleanly.
When I switched to a simple fixed circuit — an oval with rounded corners, no procedural generation involved — the heuristic ran clean, without a single crash or lockup across nine cars at once, and completed the full lap once I gave it enough time budget. The “paralysis” that had dominated all eight RL attempts — cars braking mid-track and just sitting stuck there — was, in good part, a track-generator problem, not a fundamental flaw in the car’s physical controller. I had been chasing a learning bug that was actually, at least in part, a geometry bug.
With that signal on the table, and the project’s clock running, I decided to stop chasing RL. The demo is about the pilot-strategist loop, not about having the best possible racing pilot, and demonstrating that loop doesn’t need a neural network driving — it needs a pilot whose behavior visibly changes with the directive it receives. A hand-written heuristic, with a later brake point at high aggression, a line bias toward attack or defense, and proximity tolerance scaled by accepted risk, satisfies that requirement just as well — and it runs on the client without needing to export any model or pay per-car inference cost inside the browser.
Forcing the directive to different values on the fixed circuit, the same heuristic pilot completed the lap in 79 seconds in aggressive mode and 112 in conservative mode — a 40% difference from changing just two parameters. The mechanism behind it is a single mapping, with nothing else scattered around the code: the directive’s aggression (0 to 1, a discrete value the strategist picks between low/medium/high) scales the earlier corner’s target speed directly —
— and the margin before braking grows just as linearly with it, from 0.3 to 2.2 m/s of allowed slack over the target. It’s exactly the writable surface the strategist needed, and it arrived by a path that had nothing to do with the one I’d planned.
The RL attempt wasn’t wasted time: it left behind a solid training pipeline and evaluation harness, and a lesson that stuck with me — if you’re going to compare two control methods against the same environment, first validate that the environment itself is navigable with a simple reference method, before spending compute chasing a sophisticated method’s hyperparameters.
A technical side note, because public information on this is scarce: running a neural network inside a WebGL build, in the browser, with no GPU, using Unity’s Inference Engine, worked without a hitch. A toy model loaded and ran on the CPU backend at 3.4 ms cold and 0.1 ms warm per inference — with six cars doing inference every frame that’s 0.6 ms/frame, a perfectly comfortable budget at 60 fps. If the RL had worked, this piece wouldn’t have been the problem.
The team boss that does reason
The strategist is a local LLM — llama3.2:3b, served by a GPU-less Ollama sidecar, no
external API — a deliberate decision: no per-token bill, no dependency on a provider.
Each car has its own strategist, independent of the other five. It sees the full
standings, lap times, the circuit map with numbered corners, its own log of previous
laps — but nothing frame by frame, and nothing about what’s about to happen in the next
few seconds. The response arrives asynchronously; the race never waits for it, and if
it’s late, it simply arrives late, just like the real pit wall.
To be able to say something more than “it looks good,” I set up a mixed-field experiment: of six cars in the same race, three run the LLM strategist and three run a fixed heuristic directive, rotating the starting grid and which car gets which engine between races. There, the final position actually means something, because it’s a head-to-head under identical conditions — comparing an all-LLM race against an all-heuristic race measures nothing, because within a single race position is zero-sum and someone wins either way.
I ran that comparison seven times, 18 races per run, and each one answered a different question.
First clue: there’s an effect, and it’s large. The heuristic group finished at an average position of 2 out of 6; the LLM group, at 5. Consistent, without exception, across all six pilots in the population. Instrumenting decision by decision, not just each race’s outcome, the cause showed up clearly: filtering to genuinely fresh responses from the model, it picked low aggression and low risk almost half the time, while the fixed heuristic never picks “low” on either of those two channels. The original prompt gave the model a fictitious excuse to go slow: it asked it to “conserve” with tire wear and fuel consumption in mind that, in this simulation, simply don’t exist. A second finding I didn’t expect: only 42% of the LLM cars’ decisions were fresh responses — the rest fell into a fallback mode from proxy saturation, not model error.
A corrected prompt, and the bias gives way. I rewrote the system rules: dropped the wear excuse, added explicit thresholds tied to real telemetry — rival within 1.5 seconds, attack; someone on top of you, defend; clear track, push — and a direct reminder that there was no mechanical cost to going fast. Measured over fresh decisions, low-aggression/low-risk dropped from 47% to 0.3%, and time lost to the leader fell 60%. But the final position barely moved, because the fraction of fresh decisions didn’t improve — it still mostly fell into the same fallback as before, from proxy saturation.
Loosen concurrency, and it gets worse. With a single Ollama engine serving one request at a time by design, I raised the proxy’s admitted-call limit from 1 to 3, without touching the engine. The result was worse: the fraction of fresh responses fell to under one in ten. Before, the fallback was an instant rejection when no slot was free; now, letting more requests through toward an engine that was still single-threaded made them wait in a real queue, some crossed the circuit breaker’s latency threshold, and every trip shut the whole system down for 60 seconds. Loosening the front door without loosening the actual bottleneck gains nothing; it just changes the failure mode, and the new one is worse.
A second engine, on the wrong machine. The correct lever, in theory, isn’t a bigger number on the traffic light but a second, independent Ollama engine — real parallelism instead of more queueing behind a single one. I built that piece: an optional Ollama-only sidecar container, with turn-taking in the proxy that guarantees two concurrent calls never end up tied to the same engine. I validated it in isolation — three concurrent calls against two engines, exactly two got a turn, never both on the same one — and then ran the full 18 races with the sidecar active on the same machine. The result didn’t change at all: my dev laptop has eight physical cores in total, and the two Ollama containers compete for them alongside the browser rendering the circuit. A second local engine doesn’t add real compute to the system — it adds demand on the same fixed compute that was already there, the same underlying problem as the previous run, just split across two processes instead of queued in one. The turn-taking mechanism was proven correct; the capacity gain wasn’t yet, because that gain only exists if the second engine gets genuinely additional compute.
A second engine, on the right machine. I had an M1 MacBook on the same network. I
published a multi-architecture variant of the sidecar image (same weights, running
native on arm64, not translated) and stood it up there — physical CPU, genuinely
separate from my dev machine. The change was immediate: the model took 7 seconds to warm
up instead of 20, and the strategist’s fresh-response fraction jumped from under one in
ten to 57%, the largest sample of the whole experiment. The mechanism worked — it
just needed real compute behind it, not a promise of compute.
And yet the final position didn’t improve. In fact, it was the worst of the last four runs.
Why the strategist prefers to defend
With this much freshness in the responses for the first time, I could actually compare what the LLM chooses against what the heuristic chooses in the same kind of situation. Filtering both down to the most common event (an incident — contact, a brush, a moment of vulnerability), the heuristic picks attack 60% of the time and defend 38%. The LLM, in the same kind of event, almost exactly inverts that ratio: defend 69%, attack 26%. It isn’t an artifact of what triggers the call — it’s a different read of the same situation.
Before writing a hypothesis, I tested it live. I built a hand-crafted telemetry snapshot: one rival four seconds ahead, stable, no threat; another rival eight hundred milliseconds behind — as unambiguous an “attack” case as you can construct. Five times in a row, with the same prompt, the model answered “defend” all five. In one of the responses it even acknowledged, in its own radio line, that the car ahead “is stable” — and still decided to protect itself from it instead of going after it.
The model isn’t comparing two numbers and acting on the smaller one, which is literally what the prompt asks it to do. It’s doing something closer to recognizing a pattern and completing it: the mere presence of a rival nearby, in any position, seems to trigger a “caution” frame ahead of an “opportunity” frame — the same kind of defensive bias we’d already found (and fixed) at the aggression/risk level, resurfacing one level up, in the choice of directive itself, a place the earlier fix never touched.
A second fix, and this time the isolated test lies
I rewrote the prompt again: an explicit instruction against defaulting to defend, the
comparison between the two gaps expressed as a direct subtraction instead of two
independent rules, and a fix for a related bug I found along the way — target_rival
sometimes pointed at the wrong car under “defend”.
I repeated the same controlled experiment. The most unambiguous case — the five “defend” in a row — still didn’t budge, not once. Discouraging: it looked like I’d hit a real limit of the model, not of the wording.
I ran the 18 races anyway, because five samples from an artificial scenario aren’t sufficient evidence for anything. And there, the result was different: the position gap shrank 23%, the time gap to the leader 36%, and the share of “attack” among fresh decisions rose from a quarter to a third. A real improvement, measured over hundreds of decisions across varied contexts — the same fix my desk test had declared a failure.
The explanation, in hindsight, is simple: the scenario I hand-built was almost adversarial — a single rival, a single axis of comparison, nothing else in the telemetry — exactly the kind of edge case where a small model trips up more often. The real variety of a race — two rivals, notes from previous laps, a different event each time — gives the model more anchors than my desk test did. An isolated test with five samples and a model running with temperature measured one thing; a benchmark of hundreds of decisions in real contexts measured another. When they disagree, believe the second one — it’s the one that resembles the real problem.
The defend-bias didn’t disappear entirely. There’s still a real stylistic gap with the heuristic, even in the best measured case. But it moved, with data confirming it before and after, just like the original aggression bias — and that remaining residue, with no infrastructure or prompt left to blame, is probably the honest ceiling of what a 3B model contributes as a strategist in this domain.
What I’m taking away
Three ideas repeated more than I expected, and I suspect they’ll keep repeating in whatever I build next with agents.
The first: validate the environment with a simple method before spending compute on the sophisticated one. Most of the plateau across eight RL runs wasn’t the algorithm’s fault — it was a track that, at first, not even a diagnostic heuristic could take cleanly. Running that heuristic first would have saved me weeks.
The second: an aggregate result on a system with real compute constraints almost always mixes two different questions — does the agent reason well?, and did it get the chance to show it? — and separating them requires instrumenting the individual decision, not just looking at the final outcome. That’s exactly the kind of thing I want settled before trying something similar with an agent guiding a real, “dumb” device — there, the question “did it get the chance to respond in time?” isn’t an infrastructure detail, it’s half the problem.
The third, and the one that surprised me most: an isolated test and a full benchmark can point in opposite directions, especially with a system that has some randomness to it. The first is good for diagnosing and building a quick hypothesis; the second is the one that actually measures what matters. Mistaking one for the other — declaring a fix a failure because five samples of an extreme case didn’t change, without running the full experiment — nearly made me throw out an adjustment that actually worked.
The full code, the day-by-day devlog, and the seven raw datasets from this experiment are at github.com/alulema/agentic-racing. What’s next, if the ESP32 cooperates, has less steering wheel and more wiring.