What is it?
A VAE squeezes data (like an image) into a short code, then learns to rebuild it. Because the codes are encouraged to form a smooth, well-behaved space, you can pick new codes and decode them into new, plausible samples.
Also known as: VAE, variational autoencoder, autoencoder, latent space, VQ-VAE, reparameterization trick, ELBO, beta-VAE.
Technical definition
Kingma & Welling (2013) maximise the evidence lower bound: expected reconstruction log-likelihood minus KL(q(z|x) || p(z)), using the reparameterisation trick for gradients. Outputs can be blurrier than GANs/diffusion; extensions include β-VAE (disentanglement) and VQ-VAE (discrete codes), the latter widely used as image/audio tokenisers.
Entity type: concept · Freshness class: stable.
When should I use it?
Useful for compression, representation learning, anomaly detection and as the autoencoder inside latent diffusion.
Implementation patterns
- Latent compression before diffusion.
- VQ-VAE tokenisation for images/audio.
- Anomaly detection by reconstruction error.
Technologies and frameworks
Commonly associated: PyTorch, VQ-VAE tokenisers, Stable Diffusion's autoencoder. Names are examples of what exists, not recommendations, and projects change quickly.
Practical guide
- Choose continuous vs discrete latent.
- Tune KL weight.
- Visualise reconstructions and samples.
- Use as a component, e.g. for latent diffusion.
Common failure modes
- Posterior collapse, where the decoder ignores the latent code.
- Blurry reconstructions from the pixel-wise likelihood objective.
- KL weight poorly tuned, trading reconstruction quality against latent regularity.
Troubleshooting
- Decoder ignores latent — Anneal KL weight (KL warm-up) or adjust architecture.
- Blurry outputs — Add perceptual/adversarial losses or use diffusion.
Contrasted with
- Generative Adversarial Networks (GANs) — GANs use a discriminator; VAEs optimise a likelihood bound.
- Diffusion Models — Diffusion refines noise iteratively; VAEs sample a latent and decode once.
Questions this page answers
- What is a variational autoencoder?
- What is a latent space?
- What is the difference between an autoencoder and a VAE?
- What is VQ-VAE used for?
Typical goals: understand latent-variable generative models; understand latent spaces used by image generators.
Sources and verification
Status: not independently verified. This entry was authored on 2026-10-06 from general technical knowledge, without live checking of the sources below. Treat every source as a lead to confirm (identifier, title, authorship, current version) before publication or citation.
Freshness class: stable — Core ideas that change slowly (maths, classic algorithms). Re-review yearly.
Canonical sources (unverified leads):
- Kingma & Welling, Auto-Encoding Variational Bayes (2013) — arXiv:1312.6114
- van den Oord et al., Neural Discrete Representation Learning (VQ-VAE) (2017) — arXiv:1711.00937