What is it?
Imagine a picture slowly dissolving into static. A diffusion model learns to run that film backwards: starting from random static and step by step cleaning it up into an image that matches your description. Most modern image generators work this way.
Also known as: stable diffusion, latent diffusion, DDPM, denoising diffusion, text to image model, image generation model, classifier-free guidance, score-based models.
Technical definition
Forward process adds Gaussian noise over T steps; a network (U-Net or transformer) is trained to predict the noise (or score/velocity) so that sampling can iteratively denoise from noise (DDPM, Ho et al., 2020; score-based SDE view, Song et al., 2020). Latent diffusion (Rombach et al., 2021) runs diffusion in a compressed latent space; classifier-free guidance (Ho & Salimans, 2022) trades diversity for prompt adherence; DDIM and distillation reduce sampling steps. Text conditioning typically comes from a text encoder such as CLIP or T5.
Entity type: concept · Freshness class: evolving.
When should I use it?
Use for image, audio and video generation and editing.
Implementation patterns
- Text-conditioned latent diffusion with classifier-free guidance.
- Img2img and inpainting by starting from partly noised inputs.
- Distillation for few-step sampling.
- LoRA adapters for style/subject customisation.
Technologies and frameworks
Commonly associated: Stable Diffusion family, Diffusers (Hugging Face), U-Net / DiT backbones, CLIP / T5 text encoders. Names are examples of what exists, not recommendations, and projects change quickly.
Practical guide
- Choose a pre-trained checkpoint and check its licence.
- Tune steps and guidance on a fixed seed set.
- Add LoRA for custom styles.
- Add provenance/watermarking where required (C2PA and Content Provenance (Content Credentials)).
Common failure modes
- Prompts ignored at low guidance; oversaturated artefacts at high guidance.
- Anatomy/text rendering errors.
- Training-data bias and copyright concerns.
- Slow sampling on limited hardware.
Troubleshooting
- Image ignores prompt — Raise guidance moderately; rewrite prompt; check text encoder.
- Artefacts or burnt colours — Lower guidance; change sampler/steps.
Contrasted with
- Generative Adversarial Networks (GANs) — GANs generate in one pass via adversarial training; diffusion iterates denoising and is generally more stable to train.
- Variational Autoencoders (VAEs) — VAEs encode to a latent and decode; diffusion refines noise iteratively. Latent diffusion uses a VAE for compression.
Questions this page answers
- What is a diffusion model?
- How does Stable Diffusion work?
- What is classifier-free guidance?
- Why do diffusion models take many steps?
- What is latent diffusion?
Typical goals: understand how image generators work; tune guidance and steps.
Sources and verification
Status: not independently verified. This entry was authored on 2026-10-06 from general technical knowledge, without live checking of the sources below. Treat every source as a lead to confirm (identifier, title, authorship, current version) before publication or citation.
Freshness class: evolving — Established techniques whose best practice shifts as research and tooling move. Re-review every 6 months.
Canonical sources (unverified leads):
- Ho et al., Denoising Diffusion Probabilistic Models (2020) — arXiv:2006.11239
- Rombach et al., High-Resolution Image Synthesis with Latent Diffusion Models (2021) — arXiv:2112.10752
- Ho & Salimans, Classifier-Free Diffusion Guidance (2022) — arXiv:2207.12598