What is it?
A large language model is a neural network, usually a transformer, trained on a very large body of text (and often code) to model language. "Large" refers to the number of internal parameters — commonly billions — and the amount of training data.
Its core skill is deceptively simple: given some text, estimate which token is likely to come next. Because language contains facts, logic, style and instructions, getting good at that one task produces a model that can answer questions, summarise, translate, classify, extract information and write code.
An LLM is the engine; the product around it (a chat app, a coding assistant, an agent) adds interface, instructions, tools and memory.
Why does it matter?
LLMs are the foundation of most current generative AI products. Understanding them tells you why they behave as they do:
- They are good at language and patterns, so drafting and summarising work well.
- They have a limited working memory (the context window).
- They can state falsehoods fluently (AI Hallucinations: Why Models Make Things Up), because they generate plausible text rather than looking up verified facts.
- They have a knowledge cut-off: they do not know events after their training data unless you supply that information.
How does it work?
A simplified view of one answer:
- Your text is split into What are Tokens in AI?.
- The tokens flow through the network's layers. Attention mechanisms let each token take account of the other tokens in the prompt (see What is a Transformer? (Attention Explained)).
- The model outputs a probability for every token in its vocabulary being next.
- One token is chosen (settings such as temperature control how adventurous the choice is), appended to the text, and the process repeats until the answer is finished.
Training happens in stages. Pre-training teaches general language ability from raw text. Instruction tuning and preference tuning then teach the model to follow requests and respond helpfully. Running the finished model to produce output is called inference.
Everything the model "knows" is encoded in its parameters, and the only short-term information it has is what is inside the current prompt. It does not learn from your conversation unless the provider separately retrains it.
Example
Prompt: "The capital of France is"
The model assigns high probability to " Paris", low probability to other tokens, picks one and continues. The same mechanism, repeated thousands of times, produces an email, an essay or a function. Asked for something it has no reliable pattern for — such as the title of an obscure paper — it will still produce plausible-looking text, which is how invented citations arise.
When should I use it?
Use an LLM when the task is language-shaped and a human or a check can validate the result: drafting, summarising, rewriting, extracting fields from messy text, classification, brainstorming, code assistance.
Prefer something else when you need guaranteed arithmetic or lookups (call a calculator or database through Function Calling (Tool Calling) Explained), when answers must come from your own documents (What is RAG (Retrieval-Augmented Generation)?), or when exact reproducibility is mandatory.
Common mistakes
- Assuming it has live, up-to-date knowledge. Unless connected to search or tools, it does not.
- Assuming it remembers earlier chats. Each request only knows what is sent in it; apps re-send history. See What is a Context Window? and Agent Memory: Short-Term, Long-Term and Beyond.
- Confusing the model with the product. Behaviour depends on the model and the system prompt, tools and settings around it (What is a System Prompt?).
- Judging by one good answer. Test with many realistic cases (How to Evaluate AI Systems and Outputs).