Diffusion Model
A diffusion model is a class of generative machine learning model that creates new data — most commonly images — by learning to reverse a gradual noising process, starting from pure random noise and iteratively denoising it, step by step,…
Definition
A diffusion model is a class of generative machine learning model that creates new data — most commonly images — by learning to reverse a gradual noising process, starting from pure random noise and iteratively denoising it, step by step, into a coherent output guided by a learned model and, often, a text prompt.
Overview
Diffusion models work by defining two processes. The "forward" process incrementally adds random Gaussian noise to a training image over many steps until it becomes indistinguishable from pure noise. The "reverse" process is what the model actually learns: given a noisy image at a given step, predict what noise was added, so it can be subtracted to move one step closer to a clean image. Once trained, generating a new image starts from pure random noise and repeatedly applies this learned denoising step, gradually revealing a coherent image over dozens or hundreds of iterations. Text-to-image diffusion models add a conditioning mechanism so this denoising process is guided by a text prompt rather than being unconditional: a text encoder (often a CLIP-style or dedicated language model) converts the prompt into an embedding that's fed into the denoising network at each step, steering the output toward images matching the prompt's description. Popular systems like Stable Diffusion, Stable Diffusion XL, FLUX.1, and OpenAI's DALL-E all build on this core diffusion mechanism, differing mainly in their network architecture, training data, and conditioning approach. Diffusion models became the dominant approach for high-quality image generation starting around 2021-2022, generally surpassing GANs on image diversity and training stability, though at the cost of slower generation — since diffusion requires many iterative denoising steps rather than a GAN's single forward pass, though research into faster sampling methods (such as distillation techniques used in FLUX.1 [schnell]) has substantially closed that gap. Beyond images, the same core denoising principle has been applied to audio generation, video generation, and molecule/protein design, making diffusion one of the most broadly influential generative modeling techniques in current AI research.
Key Concepts
- Generates data by learning to reverse a gradual noise-adding process
- Starts generation from pure random noise, refined over many iterative steps
- Text-conditioned variants use a text encoder embedding to guide denoising toward a prompt
- Generally surpasses GANs in output diversity and training stability
- Slower to sample than GANs due to multi-step iterative generation (though distillation narrows this gap)
- Applied beyond images to audio, video, and molecular/protein generation