In what's inside a vector, we built one by hand: describe animals with columns you chose — weight, legs, furriness — and every number means exactly what you said. You can read it like a spreadsheet.

Now open a real embedding, the kind a model produces for a word or a sentence. Same shape — a list of numbers. But it's fifty of them, or three hundred and eighty-four, and they look like this in spirit: a jumble of decimals, some positive, some negative, none of them labeled, none of them obviously meaning anything. You go hunting for the "furriness" column and there isn't one. There's no column that means anything you can name.

This is the moment embeddings feel most like voodoo. It's also completely explainable, and the explanation is the deepest thing you can understand about why vectors are shaped the way they are.

The model chose the axes, and it didn't choose them for you

When you built the animal table, you picked the columns, so of course you could read them. A real embedding is built by a model that picked its own columns — and it picked them to be useful, not readable. Those are different goals, and optimizing for one quietly sacrifices the other.

Two things happen when a model invents its own measurements:

Each idea gets smeared across many numbers. There is no single "furriness" dimension because the model had no reason to keep any one concept in one place. A human-nameable idea — royalty, plural, anger — ends up spread as a faint pattern across dozens of dimensions at once. Reading one number tells you almost nothing, the way reading one pixel tells you nothing about what a photo depicts.

Each number carries pieces of many ideas at once. Running the other direction, a single dimension isn't dedicated to anything — it holds slivers of lots of unrelated concepts stacked on top of each other. Models do this on purpose: it lets them cram far more meaning into far fewer numbers than a tidy one-concept-per-column layout ever could. The cost of that efficiency is exactly the readability you're missing.

Put those together and the "noise" makes sense. Every individual number is a blur of many meanings, and every meaning is a blur across many numbers. There is genuinely nothing to read slot by slot. That's not the model failing to be clear — it's the model being dense, which is what makes it powerful.

Same shape, but unreadable. On the left, a short hand-built vector with labeled columns weight, legs, furriness — each number readable. On the right, an illustrative learned embedding: a long row of unlabeled decimals with question marks where labels would be, because no column means anything on its own. Each number is a blur of many meanings; each meaning a blur across many numbers.

The meaning didn't vanish — it moved

Here's the part that turns the lights on. The meaning is absolutely in there. It just isn't in the axes (the individual columns) anymore. It's in the directions.

Think of an ordinary map. The horizontal axis isn't "the way to the mountains" and neither is the vertical one — but north-east is a perfectly real, meaningful direction, made of both axes together. No single axis carries it; the combination does.

Embedding space is like that, with hundreds of axes. No single dimension means "gender," but there's a consistent direction through the space that does: step that way and words get more feminine, step back and they get more masculine. Another direction means "plural." Another means "past tense." These directions are real, and they're combinations of many dimensions at once — never any single one.

The famous party trick is the proof. Take the vector for king, subtract man, add woman, and you land right next to queen. That can only work if "royalty" and "male → female" are steady directions you can travel along — which is exactly what they turn out to be. The meaning lives in how the points sit relative to each other: the distance between them, the direction from one to another. Not in any coordinate you could point at.

This is also why you never compare embeddings column by column. You compare them with geometry — how close two points are, which way one sits from another. (That's what cosine similarity measures.) A learned vector is a location on a map you read by distance and direction, not a spreadsheet row you read cell by cell.

The meaning is in the directions, not the axes. On a map whose two axes are individually meaningless, the points king, man, woman, and queen form a parallelogram, so that king minus man plus woman lands on queen. Teal arrows mark consistent directions — "royalty," "male to female," and "plural." You read a learned vector with geometry — distance and direction — never cell by cell.

The exception that proves it's a choice

To be fair, readable-number embeddings do exist — which is the best evidence that the unreadable ones are unreadable on purpose.

There's an older family of methods (topic models, and a cousin called non-negative matrix factorization) that turns a pile of documents into vectors where each number really does mean one thing. Ask for, say, 32 of them and you get 32 columns that correspond to discovered themes — one tracks how much a document is about sports, another how much it's about cooking, another politics. A document's vector is then a readable recipe: "60% cooking, 20% travel, the rest scattered." You can read those slot by slot, just like the animal table.

They're interpretable because they were built to be — the method is constrained to keep one theme per column. And they pay for it: they capture coarse topics well but miss the fine, tangled shades of meaning that a modern embedding catches, precisely because a modern embedding is allowed to smear and overlap. That's the trade in one sentence: readable and coarse, or dense and unreadable. Mainstream embedding models take the second deal every time, because for finding similar things, power beats legibility.

The readable exception. One document's 32-topic vector shown as a bar chart: cooking 0.60, travel 0.20, sports 0.05, politics 0.03. Each number tracks one topic, so it is readable — but it misses fine shades of meaning. Readable and coarse, or dense and unreadable; modern embeddings take the second deal.

Closing

So, the two questions you started with. Why is a vector shaped like a list of numbers? Because it's a set of measurements, laid out the same way for everything, so comparison is easy — the hand-built animal table showed that. Why can't you read a real one? Because a model invented those measurements to be maximally useful, which means each number is a blur of many meanings and each meaning a blur across many numbers. The legibility got traded away for density.

But nothing was lost. The meaning simply relocated — out of the labels and into the geometry, where it shows up as directions you can travel and distances you can measure. You gave up the spreadsheet and got a map. And a map, it turns out, is a far better thing to have when the question is "what else is near this?"

For the practical side of choosing and using these models, see Vector embeddings 101; for how a model comes to place things on the map in the first place, how a computer learns what words mean.