What is it?
The transformer is a neural-network architecture introduced in the 2017 paper "Attention Is All You Need" (Vaswani et al.). It replaced earlier designs (recurrent networks) that read text one word at a time, and it is the basis of most large language models, including the "T" in GPT (Generative Pre-trained Transformer).
Its central mechanism is attention: a way for each token to look at all the other tokens in the input and decide how much each one should influence its own meaning.
Why does it matter?
Two properties made transformers dominant:
- Context awareness. In "The trophy didn't fit in the suitcase because it was too big", attention helps the model link "it" to "trophy" rather than "suitcase".
- Parallel training. Because tokens in a sequence are processed together rather than strictly one by one, training can use modern hardware efficiently, which made it practical to train very large models on very large datasets.
The trade-off is that attention compares tokens with each other, so cost grows with input length. That is one reason context windows have limits (What is a Context Window?) and long prompts cost more.
How does it work?
A simplified picture, without the mathematics:
- Text becomes What are Tokens in AI?, and each token becomes a vector of numbers (an embedding), with information about its position.
- In each attention layer, every token produces a query ("what am I looking for?"), a key ("what do I offer?") and a value ("what information do I carry?"). Tokens whose keys match a query strongly contribute more of their values.
- Several attention "heads" run in parallel, each free to pick up different relationships (grammar, reference, topic).
- A small feed-forward network further transforms each token, and this attention-plus-transform block is stacked many times.
- In a text-generating model, the final layer produces probabilities for the next token.
GPT-style models are "decoder-only": each token may attend only to earlier tokens, which suits left-to-right generation. Other designs (such as BERT-style encoders) look in both directions and are often used for understanding tasks and for producing embeddings.
Example
Take the sentence "She poured water into the glass because it was empty."
For the token "it", attention scores relate it strongly to "glass" and weakly to "water", based on patterns the model learned in training. That relationship feeds into later layers, which is part of how the model resolves what the sentence means. Nothing here is hand-coded; the weighting patterns are learned.
When should I use it?
You rarely "choose" a transformer directly; you choose a model built on one. Knowing the idea helps you reason about behaviour: why order and wording in a prompt matter, why long inputs are costly, and why different model types (generative decoders vs. embedding encoders) suit different jobs.
Common mistakes
- Thinking attention equals human attention or understanding. It is a mathematical weighting.
- Assuming "transformer" means "text only". Transformers are also used for images, audio and multimodal models (What is Multimodal AI?).
- Assuming attention shows the model's "reasoning". Attention patterns are not a reliable explanation of why an answer was produced.