September 2026 · paper-1-freeze-2219-g3865bb0
This is a short account of a reasoning machine built on one consumer graphics card. It reads math word problems written in ordinary English, converts them into small diagrams of quantities and arithmetic, and solves the diagrams with exact logic — never by guessing. The interesting part is not any single component but the shape of the whole: a frozen language model that is read once and never trained; a narrow 512-number “waist” that squeezes away everything about a sentence that doesn’t matter; a notebook that carries memory across cycles of thought, and a roundabout that makes it the only road memory can take; an atlas of known operations maintained like a running census; a solver that says back, on every cycle, what it can prove and exactly where a reading contradicts itself; and a wall of independent witnesses that must all agree before the system is allowed to speak. When the witnesses disagree, the system stays silent — and its silence is information. The essay walks through the guiding images that shaped the design: happy families, a dancer’s silhouette, a compiler’s ladder of representations, a traffic circle, a fusion reactor’s twist, a wave off the coast of Portugal, and a fish with two sets of jaws.
Tolstoy opens Anna Karenina with the line: happy families are all alike; every unhappy family is unhappy in its own way. That sentence is the deepest fact we know about machine reasoning.
When our system reads a word problem correctly, the result is always the same object: one canonical diagram of quantities and the arithmetic linking them. Every correct reading is alike. But when it misreads, each failure is broken in its own particular way — a swapped argument here, a phantom quantity there, a number wired to the wrong role. Wrongness has infinite variety; rightness has exactly one shape.
This asymmetry is not a curiosity. It is the engine of the entire safety design. If you ask many independent readers to read the same problem, their wrong answers scatter — each unhappy in its own way — but their right answers collide, because there is only one happy family to land in. Agreement across genuinely independent readings is therefore evidence of truth, not just confidence. Everything else in this essay builds toward exploiting that one fact.
The grouper is a fish with two sets of jaws. The front jaws grip; the throat jaws crush. One set handles the messy, slippery encounter with the world. The other does the decisive work in a controlled space.
Our system is built the same way. The front jaw is neural: a small trained head that reads English prose — with all its ambiguity, its costume changes, its irrelevant detail — and grips it into a typed structure. The throat jaw is symbolic: an exact constraint solver that takes the structure and crushes it, by deterministic logical search, into an answer or a refusal.
The law of the house: neural proposes, symbolic disposes. The neural jaw is allowed to be creative, fuzzy, statistical — that’s what gripping wild prose requires. The symbolic jaw is never allowed to be any of those things. Abstraction may live in recognition. It may never live in verification.
How do you recognize a waltz? Not by the color of the dress. You watch the silhouette — the envelope of motion — and the dance identifies itself no matter who wears what.
Word problems work the same way. Under “Maria has three times as many apples as Ben” and “the reservoir holds triple what the tank does” is the same dance: one quantity, multiplied, equaling another. Our parser is trained to see the silhouette and ignore the costume. Internally this splits cleanly in the network’s own signals: one part of the state carries the envelope (what structure is being expressed) and another carries the carrier (which particular words expressed it). The head segments the sentence into moves and classifies each move — segment and classify, like watching a dancer — and the costume is deliberately thrown away.
Between the language model and everything downstream sits a deliberate narrowing: every token’s rich representation is forced through a 512-dimensional waist. This is Tishby’s information bottleneck made architectural. The goal of understanding is not to preserve information — it is to destroy precisely the information that shouldn’t matter (names, phrasing, order, costume) while preserving the little that must survive (quantities, roles, relations). A narrow waist is not a compromise. It is the mechanism. If two sentences mean the same thing, the waist should make them become the same thing.
Strip away the vocabulary and here is what we are really building: a lookup table with fuzzy matching and grouping. That description sounds deflationary. It isn’t — it’s the design’s honesty.
The system maintains atlases: maps of the operation-shapes it knows, in both the language space (how English expresses an operation) and the math space (what the operation does). Each entry is a centroid — the average location of a known kind — and the atlases are maintained across generations with Welford’s running statistics, the numerically careful way to keep a mean and variance up to date as new examples stream in. Seven generations of these centroids exist, one per era of the trained head, because each new generation of weights rotates the internal coordinate system — the map must be re-anchored every time (the rotation is nearly pure: aligned generations agree at cosine 0.988).
Reading a problem, then, is: fuzzily match the incoming silhouette against the atlas, group it with its kind, and retrieve the exact machinery for that kind. Recognition is lookup. What keeps this from being mere memorization is the waist: the table’s keys are silhouettes, not sentences.
In 50 First Dates, Lucy wakes every morning with no memory, and the notebook by her bed re-tells her the story so far.
The frozen language model at the base of our system is Lucy. It is read once per problem and never trained — every problem, it wakes with no memory of anything we’ve learned. All accumulated understanding lives in the small trained head, and within a single problem, working memory lives in an explicit notebook: a set of slots the head writes to and reads from across its cycles of deliberation. Facts committed to the notebook survive from one cycle to the next; everything else evaporates. Memory, in this architecture, is not a haze distributed through a billion weights. It is written down, in named slots, where you can read it.
And since this September it is the only thing that crosses from one cycle of thought to the next. That took a roundabout.
The head does not read a problem in one pass. It breathes: seven cycles of attention in which a compact state reads the sentence, forms tentative structure, reconsiders, and progressively commits. Deliberation is a loop, not a line.
Time inside the loop is kept by clocks — three of them, one per kind of structure, which we call the three-rotor stack:
Spatial, temporal, relational: where, when, what-to-what. Three hands on one clock, geared together.
The state those hands turn is itself written in polar form. Each slot’s 512 numbers are read as a direction on a 256-torus, one phase per pair of dimensions, and a radius. The clock owns sixty-four of those planes and turns them a fixed sixty degrees each breath, with no gain to learn and nothing to decline; the rest carry content. Direction is identity, what the slot is about, and radius is consolidation, how far the reading has firmed. Two readings that mean the same thing point the same way regardless of how sure either is. Writing the state this way is what makes the rest expressible: a bearing can be exempted from a cut, a memory can be written at phase zero and re-phased when read, and a rotation on the bus can never be mistaken for a rotation of the clock.
A traffic light is a gain: a learned scalar that decides how much of an organ gets through, and which training, given any alternative route, turns down until the organ is silent. We watched it happen to a two-million-parameter attention organ whose gain was born at 0.02 and closed to 0.0002. A roundabout is geometry. Nothing decides; traffic yields to what is already in the circle, and there is no way around. The parts of this system that have always worked, the waist, the mouth, the answer key, are roundabouts: each works because it cannot be bypassed. The law we wrote from that is the mandatory-road law: every organ enters as a road with no bypass, never as an invitation through a gain. And its corollary, learned from two failed experiments in one week: a road built beside a standing bypass teaches the machine to survive the road’s absence, not to use it. If you want an organ to carry, sever the alternative.
The biggest bypass in a transformer is the residual connection, the highway that carries the state from one cycle to the next, plus an update. Every memory organ can be routed around it, because the state carries itself. So we cut it, helically. Across the six cycles of deliberation the residual’s weight runs 0.93, 0.75, 0.50, 0.25, 0.07, and exactly zero, with the notebook’s read taking the complement, no cliff at any step. We named the form after the stellarator, the fusion reactor whose confinement comes from a twist built into its geometry rather than a current induced through it. The clock is exempt from the cut, since a bearing is a coordinate system and not a road. By the last cycle, nothing of the previous state crosses directly; what the final reading operates on is the notebook’s ink, addressed by what the state wanted to recall.
We measured what the residual was worth before cutting it: with the residual removed at read time, the trained machine scored 0.001 on wild problems and 0.000 on synthetic ones. Everything. Retrained under the cut, it recovered all of it and then some, and the notebook’s own value, measured by removing it, went from four points to fourteen on wild text and from sixteen to fifty-one on synthetic. The memory organ became the road because it was made the only one. A first attempt at this in August had scored zero on three hundred problems; the autopsy said it died of payload, not principle, and the autopsy was right.
A loop is not yet a cycle. We learned that the week we thought the air inside the loop was stuck. It was not: the state turns over by about half on every breath, the direction swings thirty degrees, and by the last breath not one slot has stopped moving. What we had built was a heater in a closed room. Turbulent, and going nowhere.
A heat engine needs four things, and the machine is now read against them. Reservoirs, where something persists: the notebook, holding what each slot was thinking, and the garage, holding what the state committed, bound to roles. A working fluid that moves: the state, which never freezes and needs no ratchet. A cold leg, where the fluid gives something up and becomes definite: the commit, a soft slot turned into a discrete edge for the solver, which we learned only carries when it is made mandatory, when the loop has no way around it. A return, so that what the cold leg produced comes back to where it started: the solver’s facts, and above all its refusals, re-entering the reading.
Read this way, the diagnosis was plain. The reservoirs existed. The fluid moved. The cold leg existed but was not the regime the machine ran in. And the return whispered: the solver’s facts re-entered the reading at two percent of its band and its refusals not at all. The stellarator is the cold leg built into the geometry, since by the last breath the state cannot continue except through the junction where it is written down. The steering wheel is the return. The order of the work since has been the order of the cycle.
Chris Lattner, who built LLVM and MLIR, teaches that a compiler should never jump from source code to machine code in one leap — it should descend a ladder of intermediate representations, each preserving exactly what the next stage needs. His warning became one of our laws: “Premature lowering is the root of all evil.” Commit to low-level detail too early and you destroy structure you’ll need later.
Our system is such a compiler for language: prose lowers to silhouette, silhouette to typed factor graph, recurring patterns to named macro-abstractions, macros back down to primitives at the moment of verification — every rung machine-checkable.
But there is a twist the compiler world doesn’t have. A language model writing its reasoning as text must lower left to right, one token at a time, committing early and permanently — the exact sin Lattner warns against. Our head lowers in parallel: the whole graph descends through the representations together, every part refined a little on each breath, like an image emerging from noise in a diffusion model. Nothing is forced to commit first. The parts settle jointly, and constraint flows in every direction while they do.
Deliberation gets one more structure: alternation between the two jaws during thinking, not just after it.
Each cycle, the neural jaw commits what it is confident about. The symbolic jaw takes those committed fragments and runs cheap, exact deduction — propagating consequences the way a sudoku player fills forced cells. What it derives is handed back to the neural jaw as established fact, and — just as important — the committed structure reshapes the head’s own attention: dynamic attention masking, where the parse’s current skeleton decides what the next breath is allowed to look at. Neural proposes, symbolic disposes, neural re-attends. Ping, pong.
We report this honestly, in two parts. The machinery for alternating on every breath is built and proven against the fused loop. And for months it seemed to change nothing, until we found why: what the solver handed back was injected into a part of the state that only the final readout reads. The loop itself never saw it. The solver had been talking to the mouth, not the hands. The return signal now enters the loop’s own lanes, through the mask the next breath reads under, which brings us to what the solver actually says.
On every breath it says three things. Facts: the quantities it can already determine. Refusal: this reading cannot be satisfied. And the minimal unsatisfiable core: on a refusal, the smallest set of factors that cannot coexist. Measured on real problems, the solver refuses half of all wrong readings and not one right reading in 237, and three of every four factors the core names are in fact wrong. That core is the steering wheel. The mask for the next breath opens attention among the named factors and sends them back to their sentences, so the trajectory of attention across the seven breaths is not a schedule anyone wrote. The rules write the first mask and the refusals write the rest.
The rules matter more than they sound. In June, a small frozen language model with hand-written attention masks solved Sudoku, and the same model without them scored zero. Not worse; zero. The masks were the rules of the game written as who-may-look-at-whom. Our masks come from the parse, by the same rule: allow attention where a constraint is shared. On word problems, with five factors, the mask is nearly free, because dense attention over five items loses nothing. On a puzzle with eighty-one cells, the mask is the computation. The steering wheel will be proven first where the mask is mandatory by the domain.
At Nazaré, off the Portuguese coast, an underwater canyon funnels the scattered energy of the whole Atlantic into single hundred-foot waves. The topography you cannot see is what makes the wave you can.
Language is the ocean: diffuse, vast, disordered. The architecture is the canyon. Waist, silhouette, atlas, notebook, rotors — all of it exists to funnel the scattered energy of a sentence into one narrow channel where it rises into a single, steep, breakable crest: a small exact graph that either resolves or visibly does not. You don’t fight the ocean’s variety. You shape the floor beneath it.
Iain Pears’s novel tells one story through four unreliable narrators; the title comes from Francis Bacon, who wrote of the instantia crucis — the crucial instance, the signpost that finally points one way when all others are ambiguous. No single witness settles the truth. The right collection of witnesses does.
This is our certification wall, and it is where the happy-families fact pays off. Before the system may answer, the problem is re-read five ways — the sentences permuted, the costume shuffled — and the readings must agree. Then models from different lineages and different widths, trained separately, must land on the same graph: many landscapes, one shape. Then, behind everything, an out-of-distribution “mouth” checks whether this problem even lives in territory the system knows. Independent witnesses, independent failure modes; wrong readings scatter, right readings collide. Only unanimity crosses the wall — a fingerpost assembled from testimony.
And when the solver refuses, it refuses legibly: it hands back the core, the smallest set of constraints that cannot all be true. Not “no,” but “no, and here is exactly the contradiction.” Even failure has a shape you can read, and the machine reads it on the next breath.
One honest limit of the happy-families fact, found this September. Collision works at the level of whole readings, where unanimity is a certification tier. At the level of individual factors it does not: a factor on which two independent readings disagree is wrong 78% of the time, and one on which they agree is wrong 72% of the time. On wild text three quarters of all factors are wrong, so the front between two readings is everywhere and points nowhere. Disagreement is a rumor; the solver’s core is a proof. That is why the wheel is driven by the core and the wall is built from unanimity.
All of it runs on a single AMD 7900 XTX — a consumer gaming card with 24 GB of memory — through tinygrad, a deliberately small machine-learning library that compiles its own GPU kernels and, crucially, drives the AMD card well through a lean custom driver, no vendor bloat required. The trained head is a fraction of one percent of the total system’s parameters; the frozen model does the heavy lifting exactly once per problem. The point is not thrift for its own sake. A reasoning machine whose every experiment fits on one card is a reasoning machine one person can audit, rerun, and disbelieve properly.
Honesty requires a list of the organs not yet grown. A perceiver — a dedicated organ that watches and segments raw input before the parse, once explored and set aside — may yet be missing from the current body; we are only now beginning to suspect it again. The atlas, today, lives in flat Euclidean space; kinds and sub-kinds have a tree-like structure that flat space represents poorly, and a Poincaré ball — hyperbolic space, where trees embed naturally — is the likely future home. The words never change: after the frozen model, the trained representation of a word is a single linear map, there is no trained attention among the words, and no road updates a word in light of what the slots have decided, so the words the machine reads on its seventh breath are the words it read on its first. Two layers on that side are registered and unbuilt. And the bus that binds what-to-what writes its deposits at whatever bearing the clock had when they were written; a canonical frame for the memories is being tested as this is written. The ledger records what each of these must demonstrate before it earns its place. Most ideas don’t. That’s the point of the wall.
A frozen giant read once. A waist that destroys the right information. A silhouette instead of a costume. A notebook instead of a haze, and a roundabout that makes it the only road. Three clocks: where, when, what-to-what. A compiler that refuses to lower too early and lowers everything in parallel when it does. Two jaws alternating on every breath — grip, crush, grip again — with the crush’s refusal steering the next grip. A canyon that funnels an ocean into one wave. And a wall of witnesses with one rule: all the happy families look alike, so speak only when your readings collide — otherwise, stay silent, and make even the silence legible.
That is the shape of thought we are building. Not a bigger mind — a more honest one.
— Bryce Roche & Claude, September 2026