The transformer block: putting the pieces together

Transformer on-ramp · Part 8 of 8

Now we put the pieces together. A transformer is not a mystery: it is one small block repeated many times. If you followed Parts 1 to 7, you already know every part of it.

The recipe for one block

You now have all the main ingredients. One transformer block combines them in a fixed order:

1. LayerNorm. Tidy the numbers by normalising across the features of each token (not across the batch).
2. Multi-head self-attention. Tokens exchange information.
3. Residual add. Add the original input back: x = x + attention(...).
4. LayerNorm again, then a small MLP applied to each token separately. It widens to four times the model width, applies a smooth non-linearity (GELU) and narrows back.
5. Second residual add. x = x + mlp(...).

Normalising before each sub-layer, as in the diagram, is called pre-norm. It is the modern default because it trains more stably than the original paper’s post-norm arrangement.

# One pre-norm block, in two lines of logic
def forward(self, x):
    x = x + self.mha(self.ln1(x))[0]    # tokens exchange information
    x = x + self.mlp(self.ln2(x))       # each token is processed on its own
    return x

Why the residual connections matter

The residual add gives the signal a short path straight through the block. In a measurement run for the course, a plain stack of 24 layers with no residuals let the gradient reaching the input shrink until it underflowed to zero. With residuals, learning signals can travel through dozens of blocks.

Stack it, and choose a direction

Stack N of these blocks and you have a transformer. Two flavours exist: an encoder, where every token may look at every other token, and a decoder, which uses the causal mask from Part 5 so each token can only look backward. Nearly every large language model you will use is a decoder-only transformer. The picture in this series is the whole engine.

Takeaways

• A block = layer norm, multi-head attention, residual add, layer norm, MLP, residual add.
• Pre-norm (normalise before each sub-layer) is the stable modern default.
• Residual connections keep gradients alive through deep stacks.
• Stack N blocks; with a causal mask this is the decoder-only design behind every LLM you will use.

That completes the series. Chapter 1 of the course goes further with a full hands-on lab, a self-test and an optional deep-dive.