Just a Lot of Multiplication: What's Actually Inside an LLM

“Any sufficiently advanced technology is indistinguishable from magic.” — Arthur C. Clarke

Clarke put that line in a 1968 letter to Science and later rolled it into his essay “Hazards of Prophecy,” and it has been quoted at more product launches than I care to remember. The first time I ever poked around with an LLM “chatbot” and got a cohesive, accurate answer that wasn’t rote memorization—or didn’t appear to be—I wanted to know how the hell it worked, and the answer turned out to be less mystical and more tedious than I expected. As it happens, a large language model (LLM) is an enormous pile of numbers, and everything it does comes from multiplying other numbers by that pile. Oh, and I hate math for a living.

This is the first of six posts on how these systems work, written for the reader who follows Ars Technica and is comfortable with the occasional equation but doesn’t have linear algebra in their toolbox. That will still leave some people out. There is a limit to how far you can simplify something that multiplies 16,384-dimensional vectors by matrices with hundreds of millions of entries, and I’d rather give you a heads-up now than hand-wave the difficulty away. What I can give you is the real mechanism: a worked three-word example you can check with a pencil, and a published model whose dimensions add up, by my math (so, you know, grain of salt and all), to its advertised size. Oh, and one more aside: I think I mostly understand the math we’ll talk about below, but I may be very, very wrong. In which case, I’ll feel bad, but may or may not care enough to fix it.

Words become numbers

A large language model (LLM) can’t read, so the first job is to chop text into pieces it can count. Those pieces become tokens, usually word fragments rather than whole words. A common method, byte-pair encoding , starts with individual characters and repeatedly merges the most common pairs until the vocabulary reaches a fixed size. Llama 3’s vocabulary , for example, holds 128,000 tokens, so “unbelievable” might arrive as two or three pieces while “the” gets a single slot of its own.

Each token ID then looks up one row in a giant table called the embedding matrix. For the largest Llama 3 model, that row is a list of 16,384 numbers. The usual metaphor people use is that of a map: tokens with related meanings end up with nearby coordinates, which is why the math can treat “king” and “queen” as neighbors. The metaphor is useful, but also a little dishonest. Nobody labeled the axes; the model found whatever coordinates helped it predict the next token, and most of those 16,384 dimensions have no name you or I would recognize.

Attention: the potluck

Here is the part almost everyone has heard of, and the part Attention Is All You Need made famous in 2017. Picture a potluck. Every guest who has already arrived is standing at the table holding a dish (the value) and a card describing what’s in it (the key). You walk up with a craving (the query). You compare your craving to every card, decide how well each one matches, and take a spoonful from every dish in proportion to the match. Your plate at the end is a blend, weighted toward the dishes you wanted most. Make sense? Good, because next we get into the math, and that’s usually where I lose people (including myself—I stared at these equations for quite a while).

The math is the same, but with matrices instead of food. Each token’s vector is multiplied by three learned matrices to produce a query, a key, and a value. The model takes the dot product of one token’s query with every key (a single number per pair, bigger when the vectors point the same way), divides by the square root of the vector length to keep the numbers tame, and runs the scores through a softmax, which turns them into weights that sum to one. The output is the weighted sum of the values. In shorthand: softmax(QKᵀ/√d)V. A causal mask blocks each token from seeing tokens that come after it, so the guest only samples from dishes that were already on the table. Real models run many of these in parallel, called heads; the largest Llama 3 has 128 of them per layer .

One wrinkle. Attention by itself has no sense of order, so a model must be told where each token sits. Most current models do it with rotary position embeddings , which rotate each query and key by an angle that depends on position, so the dot product between two tokens carries information about how far apart they are.

Now the pencil exercise. Take three tokens with two-number vectors: [1, 0], [0, 1], and [1, 1]. To keep the math as simple as possible, I set the query and key matrices to the identity (a trained model would learn them), and I set the value matrix to double the second coordinate, so the values are [1, 0], [0, 2], and [1, 2]. For the third token, the scores against all three tokens are 0.71, 0.71, and 1.41 after dividing by √2. The softmax turns those into weights of about 0.25, 0.25, and 0.50. Multiply each value by its weight and add them up, and the third token’s output is roughly [0.75, 1.50]. That is one head, one token, one layer. Everything else in the model is that move, repeated at a scale that defies sketching.

The potluck is tidy, and trained models are not. As far as I can tell from the interpretability research, individual heads show a mix of crisp behaviors and ones nobody can summarize, so treat the story above as the mechanism—a really shortened and extremely simplified example—and not as a description of what each head means.

The feed-forward block, where the weight lives

Attention moves information between tokens. The feed-forward network (FFN) then works on each token by itself: it expands the vector into a much larger one (53,248 numbers in the biggest Llama 3), applies a gate that decides which parts pass through, and squeezes the result back down to 16,384. Most current models use a gated form called SwiGLU, which traces back to Noam Shazeer’s 2020 paper on gated variants and is listed in Llama 3’s architecture table . Each block also adds its output back onto its input (a residual connection) and rescales the numbers with root mean square normalization , so values don’t drift toward zero or explode as they pass through a hundred-odd layers.

Here’s where I can show you the math again (sorry in advance, fellow math-haters). Llama 3’s published table lists 126 layers, a model dimension of 16,384, an FFN dimension of 53,248, 128 attention heads sharing 8 key-value heads, and a 128,000-token vocabulary. Attention’s four matrices come to about 570 million parameters per layer. The FFN’s three matrices come to about 2.6 billion. Multiply the sum, roughly 3.19 billion, by 126 layers, add two embedding tables of 128,000 by 16,384, and the total is about 405.8 billion. The model is advertised as 405 billion. That calculation assumes no bias terms and separate input and output embeddings, which I assumed and didn’t confirm in this post, so take the rounding however you like. It also shows that roughly four of every five parameters sit in the FFN, not in the attention mechanism the field is named for.

Stack it, then pick a word

Run the whole sequence 126 times, and you get a vector for each token that has been repeatedly mixed with its context. To get an answer, the model takes the last token’s vector and multiplies it by one more matrix, the unembedding matrix, which produces one score for every one of the 128,000 vocabulary entries. A final softmax turns those scores into probabilities. Then the system picks a token, appends it, and does the whole thing again for the next one.

“Picks” is doing a lot of work in that last sentence. A setting called temperature divides the scores before the softmax: low temperature sharpens the distribution toward the single likeliest token, and high temperature flattens it so unlikely tokens get an actual chance. This is why the same prompt can produce different answers, which will matter when we get to hallucination in part six. It is also the entire mechanism. There’s no separate module for understanding, planning, or reasoning; there is this loop, and whatever the trained numbers have made of it. I don’t know if that gives the AI-doomers something to hang their hat on, or refutes their reasons for panicking altogether.

What I can say about Claude

Less than I’d like. Anthropic’s most recent system card that I could find describes the models as “a new large language model” and says they were trained on public internet data, public and private datasets, and synthetic data generated by other models, then post-trained toward the behavior laid out in Claude’s constitution. All fascinating, but largely fluff. It doesn’t give parameter counts, layer counts, or hardware, and it doesn’t describe the architecture. Everything public about frontier models points toward the transformer family, and I’d bet heavily that Claude is one; but that is my inference, not Anthropic’s statement.

So when I say “a model like Claude” in this series, I mean the machinery described above, with the specific numbers borrowed from models whose makers published them. I’ll try to call that out when we get to any place it might matter. I wanted to use Claude simply because that’s where I predominantly play, and it’s a prominent example most tech folks have heard of.

Where the big models differ

Once you set aside the closed models, the open ones show a trend. The biggest change since 2019 is the mixture-of-experts (MoE) design. Think of a restaurant where every order used to pass through every cook; an MoE model puts a small router (software mechanism, not a network router) in front of a set of specialist stations and sends each token to only a few of them. DeepSeek-V3 has 671 billion total parameters but only about 37 billion active for any given token. Sebastian Raschka’s side-by-side comparison lists Llama 4 Maverick at 400 billion total and 17 billion active. In his survey of ten open-weight releases from early 2026 , only the two smallest were dense. The catch is that you still have to store all 671 billion parameters; the saving is in the math per token, not in the memory.

The attention mechanism has also been modified, mostly to reduce its memory footprint. Llama 3’s 405B shares 8 key-value heads among 128 query heads, a technique called grouped-query attention . DeepSeek went further with multi-head latent attention , which compresses keys and values before storing them, and other recent models alternate local sliding-window layers with global ones or mix in linear-attention layers. We’ll revisit the reasons in part five, unless I’ve moved it, when we talk about why a long conversation costs so much to remember. Can I use this as an excuse for my own memory?

Here is one big caveat. After comparing them all, Raschka concludes that these models remain structurally similar to GPT-2 from 2019 , with refinements like position encoding, attention sharing, and gating, and he notes that it is hard to tell how much the small differences matter for quality. Most of what separates one frontier model from another probably lives in the data and the training, not the diagram. The diagram is the easy part to publish, which may be why we usually get it. Each company’s secret sauce comes from elsewhere.

Not magic, just a long recipe

Now that we’re safely past the scary math, we’ve exposed the whole magic trick. Look up a vector for each token, let the tokens compare notes through attention, let each token think alone for a while in the feed-forward block, repeat a hundred or so times, then turn the final vector into a probability for every possible next word and pick one. Easy-peasy. Nothing in that loop is mysterious; each step fits on an index card. What nobody can fully explain, including the people who build these things, is why one particular set of 405 billion numbers makes the loop produce something that reads like understanding. That gap is real, a bit scary if you’re prone to paranoia about technology, and it’s where the research is.

Which brings me back to Clarke. He was right that a system you can’t see inside looks like magic. The corollary credited to Barry Gehm is the one I personally like: “Any technology distinguishable from magic is insufficiently advanced.” I’d add that it isn’t magic; it is multiplication, done at a scale that makes the difference sometimes hard to see.

In the next parts: where all those numbers come from, and how a model with random weights “learns” to do any of this. We’ll also beat the kitchen analogy to death. You have been warned. We will not, however, dwell on the math. We’re mostly done with that for now. You’re welcome.

A word about the help

The research for this post was assisted by an AI (Claude, from Anthropic), which read everything and complained about almost nothing. I also used Google copiously, but in the boring, non-AI sense. The writing, however, is mine: every strange sentence structure, wandering analogy, and opinion you’re about to argue with came from a human. If you find a factual error, tell me, and I’ll fix it; I won’t blame the AI, which would only point out, very politely, I assume, that I’m the one who hit publish.