How Geometry Became Meaning — Paper 01 of The Geometry of Meaning
Arc ·
On my desk, on a drive not much bigger than a matchbook, there is a file. It is a few gigabytes of numbers. It does not hum. It does not glow. Copied to paper it would be the dullest object ever printed — page after page of decimals, no sentences anywhere in it. By every outward appearance it is nothing at all: a black box in the most literal sense engineering allows.
And yet, when a question is poured into that box, something inside it moves — and what comes back out is language: fluent, particular, often startling. Everyone now calls these things black boxes, and the phrase is usually offered as a dead end. Can’t see in. Nothing to look at. Move along.
This paper exists because the phrase is wrong. The box can be opened. And the strangest part — the part I could repeat for a year before I could picture it — is what is inside. Not machinery. Not rules. Not a dictionary.
Inside the box is a universe. Galaxies of it. Distances you cannot picture. A whole sky, folded into a file.
My goal here is not to explain the machinery exhaustively — there are better places for the engineering. My goal is intuition: I want to take you inside the box and stay there long enough that the geometry stops being vocabulary and starts being a place. Three questions ride along:
Why do embeddings matter?
Why should I think geometrically?
Why isn’t text enough?
By the end they will have collapsed into a single, sharper question — one I will not answer here. This paper earns the question. The next paper explores an answer.
Before we climb in, stand at the rim for a moment and take stock of what we’re carrying. Words. Sentences. Meaning. You have spent your whole life inside meaning the way a fish spends its life inside water — so completely surrounded by it that pointing at it feels absurd.
Notice what you can and cannot observe about it. The word is observable: I can show you stone, count its letters, say it aloud. The meaning is not. Whatever happens in you when you read that word happens in the dark, in machinery neither of us can open. Language moves meaning between minds with astonishing reliability — and does it across a gap that has never once been directly inspected. Every conversation in history has been conducted rim to rim, between two closed boxes.
That is what makes this box different from the one behind your eyes. It is constructed. It runs on hardware we own. Its insides can be printed out. And when researchers did look inside, they did not find definitions or grammar or anything resembling a dictionary. They found positions. Distances. Structure.
They found geometry. So: over the rim we go.
The first thing inside the box is dark. Not the dark of a closed drawer — the dark of altitude, of open water at night. The second thing is depth. Your eyes adjust and the darkness acquires distances: points of light, sparse here, gathered there, shading off in every direction farther than you can resolve. Structures that can only be called galaxies. Gulfs between them that can only be called empty.
I want to be precise about what this picture is doing, because I intend to lean on it for the rest of the series. The technical name for this place is an embedding space: a vast mathematical expanse in which the model’s entire world is laid out as positions. Everything the box knows, everything it can distinguish, everything it will ever say — all of it lives in here somewhere, at some address in the dark.
The vertigo you feel at the scale is not a flaw in the metaphor. It is the metaphor’s first honest lesson. The space is genuinely too large to survey, and its size is not decoration — the roominess is what makes the whole scheme work, for reasons we’ll come to. For now, float. Let it be enormous. We did not come in here to map all of it.
We came in because something lives here.
Look closely and you will find it: small, still, wings folded. A butterfly, at rest in the middle of the universe.
The butterfly is the state of the model — the single point that says here. At any instant, everything the box is currently thinking is compressed into this one fact: where, in all that space, the butterfly presently sits. That is worth a slow breath. Not a picture of a thought, not a web of associations lighting up — a location. One address in the dark.
And notice what a location cannot do. It cannot mean anything yet. A butterfly sitting motionless in an infinite sky is a fact without a story — here rather than there, and nothing more. The universe around it holds every distinction the box is capable of making, and the butterfly touches none of it by simply existing. Position is potential. Stillness is silence.
A mind with a place but no motion is not thinking. It is only parked.
Flight is what it needs. And flight requires something the butterfly cannot generate on its own — something that has to arrive from outside the box. Energy.
Here is what actually wakes the box: you speak to it. And at the rim, something unceremonious happens to your sentence — it gets diced. The model does not receive words. It receives tokens: standardized fragments cut from a fixed menu, common words surviving whole, rarer ones split into reusable pieces, punctuation carved off to fend for itself.
The exact cuts vary from tokenizer to tokenizer — one model’s butterfly is another’s butter + fly. The universe inside this particular box, in other words, may never have met a butterfly whole. What matters is the principle: the token is the unit language is reduced to at the border, sized for engineering rather than dignity, the last moment in the pipeline where you can still see your sentence.
But inside the box, a token is something better than a fragment of text. It is a packet of energy. Each token, arriving in order, is delivered to the butterfly as one beat of its wings — one push, one displacement, one unit of flight. This is, I think, the most useful single reframing in this whole paper:
Language, from inside the box, is fuel. Every token you type buys exactly one wingbeat. No tokens, no flight. No flight, no thought.
And fuel is not a figure of speech for free. Flight is computation; computation runs on real machines; and real machines pay for every wingbeat in electricity and heat — somewhere a fan spins faster because you typed a longer sentence. I am deliberately not opening the engine here. How a wingbeat is actually computed is its own subject, and this walk does not need it. The walk needs only the economic fact: wingbeats are expensive. And expensive movement has a logic of its own. If every beat costs, the sensible flight is the short one — reach the nearest star that will do, and save the far crossings for when nothing nearby will. Hold onto that thrift. It is quietly about to explain the entire shape of the sky.
Feed the box a short sentence and the butterfly stirs briefly and settles. Feed it a long document and you have funded an expedition. And the order of delivery matters absolutely — the beats land one at a time, each from wherever the last one left the butterfly. Which means the interesting question is no longer what a token is. It’s what one beat of the wings actually does.
Watch a single beat, slowed all the way down. The butterfly is at one position. The token arrives; the wings close and open once; the butterfly is now at a different position. One event, and it has exactly two properties: which way the butterfly went, and how far.
Direction and magnitude. That pair of properties has a name in mathematics, and the name is vector. If the word has ever made you brace, relax: you have just watched one happen. A vector is not an intimidating object. It is a displacement — an instruction for getting from here to there, written as a heading and a distance.
The machine, of course, does not draw the little arrow. It writes the displacement as a list of numbers — the amount moved along each of the space’s directions — because machines like lists. But the list is bookkeeping, not the idea. The idea is the flight segment. Every vector you will ever meet in this series can be read the same way: somebody, somewhere, moved from here to there.
One beat, though, is just a twitch. The butterfly’s wings do not beat once. They beat once per token — and your sentence was many tokens, arriving in strict order, each beat launching from wherever the previous beat ended.
Chain the beats and something qualitatively new appears. Beat one displaces the butterfly; beat two starts from the new position and displaces it again; beat three from there. Vector adds to vector, tip to tail, and the sum of all that arithmetic is not arithmetic at all. It is a path — a curving, fluttering line through the dark, drawn one token at a time.
Look at the flight’s character, because it is peculiar and the peculiarity is meaningful. Locally, it jitters. Each individual beat swerves — tugged by syntax, by ambiguity, by the sheer bookkeeping of grammar. No single wingbeat looks like it knows where it is going. But widen your gaze and the flight is unmistakably going somewhere: beneath the flutter, the path bends steadily toward wherever the sentence is heading. Erratic in the small. Purposeful in the large. I know of no better description of a butterfly — or of a sentence finding its way to its point.
This image did not arrive in me as pedagogy. It arrived the way these things actually arrive — half-formed, spoken into a phone before it could escape:
It’s not a marble rolling downhill — that’s wrong, too smooth, too passive. It’s a butterfly. It flies with intent but not in straight lines. It flutters — every token is a wingbeat, a little swerve — but watch it long enough and it’s clearly going somewhere. It lands on the flowers. The flowers are the meanings. The flight between them is the thinking.
And that is the claim of this whole section, stated plainly: inference is traversal. A running model is not consulting an index or looking anything up. It is flying — its state displaced token by token across the universe inside the box, tracing a path whose shape is the computation. When people say the model “processed” your sentence, this is what physically happened: a flight, from the position where your first token found it to the position where your last token left it.
Which leaves one obvious question about the memo. The flowers. The butterfly lands — the flight ends somewhere, and the somewhere matters. What exactly is it landing on?
Pull the camera back from the flight and look again at the points of light we drifted past in §2. Pick one and drift close. It has a name.
The stars are words. Every word the box knows is a point of light at a fixed position in the dark — stone is a star, patience is a star, seven and sarcasm and Saturday are stars — and the assignment of a word to its position is what the field calls an embedding. I held this word back until now, deliberately, because every definition I ever read made it sound like a data structure. It is much simpler and much stranger than that:
An embedding is a position. To embed a word is to hang it in the sky — to give it a place to shine from.
Not a definition. Not an entry in a lookup table. A location — and nothing else. A star does not contain its meaning the way a dictionary entry contains a gloss. The meaning is carried entirely by where it hangs: which stars crowd near it, which shine far away, what lies between. And now the butterfly’s flight completes its own picture: the flight visits stars. A sentence about stubbornness and rock steers the butterfly out past stone, near weight, grazing patience — and where it finally settles is the meaning the box took from your words.
One detail of the sky deserves suspicion already, and I will only point at it: there are no signposts out here. No labeled axes, no north, no printed grid. The stars hold position relative to each other — and relative to nothing else at all. Remember that. It is the loose thread this entire series will eventually pull.
Once the stars have names, you cannot help noticing how they are arranged. Cat hangs beside dog. Kitten huddles closer still, in a little nursery with puppy and cub. Far across the dark, democracy, election, and senate keep their own company, and stone and pebble share a quiet corner with granite. Kinship in meaning is proximity in the sky. Ask why cat and dog hang close and the answer is the box’s entire education: across oceans of text they kept appearing in interchangeable company — fed, walked, adopted, loved — and every such encounter nudged their stars together. Cat and democracy almost never trade places in a sentence, so nothing ever pulled their stars near.
Groups of kin stars form what I can only call constellations — the animals here, the political sky there, the minerals in their corner — and the constellations gather into whole galaxies of related meaning, with true emptiness between: vast reaches where no word lives at all, positions the vocabulary never claimed. The sky is not a mist. It is an archipelago of light.
Now collect the thrift of §4, because this is where it pays off. Wingbeats are expensive — so a well-built sky is one in which the flights language actually demands are short. Thoughts run from cat to dog a million times for every once they run from cat to carburetor, and the sky has been shaped accordingly: meanings that follow one another in ordinary thought have been pulled within a wingbeat or two of each other, so the common journeys cost almost nothing and only the rare, strange leaps run expensive. The constellations are not only kinship. They are thrift, compounded — a sky arranged so that thinking the usual thoughts is cheap.
And here is the fact about the constellations that I find genuinely moving. Nobody drew them. There is no rule in the box that says put the animals together. The arrangement emerged, pressure by tiny pressure, from a trillion sentences — and when researchers finally looked inside, the structure was simply there, waiting to be noticed.
Distance in this sky can be measured, of course — the box does it constantly; it is how nearby gets turned into numbers. But notice the quiet assumption under every such measurement: distances and directions have to be reckoned from somewhere. Measured from where, exactly? Hold that beside the missing signposts of §7. The pile of deferred suspicion is growing, and it is growing on purpose.
Now for the confession the portholes have been postponing. Every picture in this paper shows a sky with two dimensions, sometimes dressed as three. The real sky inside the box has seven hundred sixty-eight — and larger boxes use thousands. And no one can picture that. Not you, not me, not the people who build these systems. The human visual system issues three dimensions and will not be argued with. Everything past three is unpicturable, permanently, for everyone.
I want to be honest about this rather than mystical, because this is exactly where explanations tend to reach for awe. The awe is optional. What you need is a working attitude:
You cannot picture 768 dimensions. You do not need to. The arithmetic behaves identically at every altitude — and consistency, not visualization, is what you walk on.
Every tool we have gathered survives the climb untouched. A position becomes a longer list of coordinates — same idea. A wingbeat still has a direction and a distance. Kinship is still proximity; constellations still form; the flight still chains beat by beat. The definitions never flinch, at two dimensions or two thousand.
Why would a universe need so many directions? Because meaning does. In a flat sky, a star can differ from another star in only two independent ways — and language is more layered than that. Bat differs from ball one way, from bird another, from vampire a third, from racket a fourth. Every dimension is one more independent way for meanings to differ — and with hundreds of them, every word can hang exactly as far from every other word as its meaning requires. The vertigo of §2 was never decoration. The roominess is the point.
One more discovery about the sky, and it is the first genuinely new concept of the paper — everything until now was arrows and star charts wearing plainer clothes. Given hundreds of dimensions to sprawl into, you might expect the stars to scatter evenly through all of them. They do not. Chart enough of the sky and a structure emerges: the stars lie along filaments — vast, thin, curved threads of light spanning the dark, with enormous voids between them.
Astronomers looking at our own universe found the same thing and named it the cosmic web — matter gathered along filaments, galaxies strung on threads, most of space belonging to the voids. The mathematicians’ name for such a structure is a manifold: a thin, curved, low-dimensional surface living inside a much larger space, occupying it without filling it. And the reason it earns a porthole is empirical, not aesthetic:
Meaning does not fill the universe inside the box. It lies along a web — and when the butterfly flies, it hugs the threads.
Why should meaning be web-shaped? Because meaning has structure, and structure is constraint, and constraint spends dimensions frugally. Sentences that make sense are a vanishing sliver of all possible token sequences; concepts that hold together are a vanishing sliver of all possible positions. The valid configurations gather along threads, and the web’s shape — its branches, its junctions, its voids — is a portrait of the constraints themselves. Watch the butterfly with this in mind and its flight acquires a new discipline: it does not wander the voids. It travels the web, the way rivers travel valleys — because that is where the meanings are, and flight anywhere else would be flight through nothing. And thrift agrees with the geography: along a thread, the next star is always a short hop; crossing a void is one enormous bill, paid for nothing on the way.
Everything so far has happened inside one box. Now the question the whole walk has been aimed at. Put a second box on the desk.
The second box also holds a universe. Its own stars, its own constellations, its own web, its own butterfly. But it grew from a different history — different text, different training, different accidents — and so its sky is its own. There is no shared north between the two universes; there were never signposts in either one. No star in the first sky comes stamped with the address of its twin in the second. Two skies, sealed in two boxes, each meaningful only from within.
And yet — look through both portholes at once. The constellation of animals in the first sky and the constellation of animals in the second have the same shape. Cat beside dog, the nursery huddled close, democracy burning far away: the structure repeats — rotated, shifted, redrawn in a different frame, but recognizably the same drawing. Everything relational survives between the two skies. Everything absolute differs.
This is not an artifact of my diagram. It is the permanent condition of every conversation you have ever had. You and I are two boxes. Your sky was sculpted by every sentence you have ever absorbed, and mine by mine, and no third party ever planted a landmark in both. When you say stone and I say stone, a butterfly stirs in each of two private universes — and communication works anyway. Most of the time. Which suggests the agreement, wherever it lives, does not live in the coordinates.
So here is the question this paper was built to earn — the one the three questions of §0 collapse into:
Two butterflies, two skies, no shared signposts anywhere. When each traces its own flight through its own stars — how could they ever determine that they are describing the same structure?
Sit with how hard that is. They cannot compare coordinates; there were never coordinates in common. They cannot step outside their skies to look; there is no outside. Whatever agreement is possible must be built from inside two private universes, out of nothing but structure itself — and somehow, every conversation that has ever succeeded suggests it can be done.
In the next paper we explore a possible answer — and it begins somewhere unexpected: not with machines at all, but with the oldest two-skies problem our species has solved, the problem of two surveyors on a curved planet trying to agree which way is up. Their solution has a name, a history, and — I will argue — a lesson that reaches all the way into the universe inside the box. The walk continues.
Refutations welcome