Recap: Denoising Diffusion Implicit Models (DDIM)

While Denoising Diffusion Probabilistic Models (DDPMs) provide a powerful framework for high-quality image generation, their iterative sampling process, often requiring hundreds or thousands of steps, presents a significant computational bottleneck. Denoising Diffusion Implicit Models (DDIMs) were introduced as a generalization of DDPMs specifically designed to address this limitation by enabling much faster sampling.

A significant departure from DDPMs lies in the nature of the reverse process. DDPMs assume a Markovian reverse process, meaning $p_\theta(x_{0:T}) = p(x_T) \prod_{t=1}^T p_\theta(x_{t-1}|x_t)$, where each step $x_{t-1}$ only depends on the previous step $x_t$. DDIMs, however, leverage a more general, non-Markovian inference process. This seemingly small change has profound implications. Importantly, DDIMs utilize the exact same neural network, $\epsilon_\theta(x_t, t)$, trained with the standard DDPM objective (often the simplified version predicting noise). The innovation is entirely within the sampling procedure.

The DDIM Sampling Process

The core idea behind DDIM sampling stems from analyzing the conditional distribution $q(x_{t-1}|x_t, x_0)$ used in the DDPM forward process derivation. Recall that in DDPM, the reverse step $p_\theta(x_{t-1}|x_t)$ aims to approximate $q(x_{t-1}|x_t)$. DDIM instead designs a sampling process that directly uses properties derived from $q(x_{t-1}|x_t, x_0)$.

First, we can obtain an estimate of the initial data point $x_0$ given $x_t$ and the predicted noise $\epsilon_\theta(x_t, t)$. Using the forward process definition $x_t = \sqrt{\bar{\alpha}_t} x_0 + \sqrt{1 - \bar{\alpha}_t} \epsilon$ (where $\epsilon \sim \mathcal{N}(0, I)$ and $\bar{\alpha}_t = \prod_{i=1}^t \alpha_i = \prod_{i=1}^t (1-\beta_i)$), we can rearrange to predict $x_0$:

$$\hat{x}_0(x_t, t) = \frac{1}{\sqrt{\bar{\alpha}_t}} (x_t - \sqrt{1 - \bar{\alpha}_t} \epsilon_\theta(x_t, t))$$

This $\hat{x}_0$ represents the model's best guess of the original clean image given the noisy image $x_t$ at timestep $t$.

Now, instead of sampling $x_{t-1}$ from the approximate posterior $p_\theta(x_{t-1}|x_t)$, DDIM defines a direct sampling step using $\hat{x}_0$. The full DDIM update step, considering a subsequence of timesteps $t_i, t_{i-1}$ (where $i$ goes from $S$ down to 1, and $t_S=T, t_0=0$), is given by:

$$x_{t_{i-1}} = \underbrace{\sqrt{\bar{\alpha}_{t_{i-1}}} \hat{x}_0(x_{t_i}, t_i)}_{\text{Direction to } x_0} + \underbrace{\sqrt{1 - \bar{\alpha}_{t_{i-1}} - \sigma_{t_i}^2} \cdot \epsilon_\theta(x_{t_i}, t_i)}_{\text{Direction of noise}} + \underbrace{\sigma_{t_i} z_t}_{\text{Random noise}}$$

Here, $z_t \sim \mathcal{N}(0, I)$ is fresh Gaussian noise, and $\sigma_{t_i}^2$ controls the stochasticity of the process. It's typically parameterized by $\eta$:

$$\sigma_{t_i}^2 = \eta \frac{(1-\bar{\alpha}_{t_{i-1}})}{(1-\bar{\alpha}_{t_i})} \left( 1 - \frac{\bar{\alpha}_{t_i}}{\bar{\alpha}_{t_{i-1}}} \right)$$

Deterministic Sampling and Acceleration

A major feature of DDIM arises when setting the hyperparameter $\eta = 0$. This makes $\sigma_{t_i} = 0$, eliminating the random noise term $z_t$ and resulting in a deterministic update rule:

$$x_{t_{i-1}} = \sqrt{\bar{\alpha}_{t_{i-1}}} \hat{x}_0(x_{t_i}, t_i) + \sqrt{1 - \bar{\alpha}_{t_{i-1}}} \cdot \epsilon_\theta(x_{t_i}, t_i)$$

This deterministic nature means that starting from the same initial noise $x_T$, the sampling process will always produce the exact same final image $x_0$. This property is valuable for tasks requiring reproducibility or manipulation of the latent space.

Furthermore, the non-Markovian formulation allows DDIM to skip steps during sampling. While DDPM typically requires sampling across all $T$ timesteps (e.g., $T=1000$), DDIM can use a much shorter subsequence $\tau = \{t_1, t_2, ..., t_S\}$ where $S \ll T$ (e.g., $S=20, 50,$ or $100$). The sampler jumps directly from $x_{t_i}$ to $x_{t_{i-1}}$, significantly reducing the number of required forward passes through the network $\epsilon_\theta$.

Comparison of sampling paths for DDPM (top, blue) and DDIM (bottom, red). DDIM allows for significantly fewer steps ($S$) compared to the original number of diffusion steps ($T$).

Trade-offs and Connections

The speedup offered by DDIM comes with a trade-off. While significantly faster, using fewer sampling steps ($S$) can sometimes lead to a slight reduction in sample quality or diversity compared to running the full DDPM process or using a larger $S$. The choice of $S$ and $\eta$ allows tuning this balance between speed and fidelity. Setting $\eta=1$ recovers a process closely related to the original DDPM sampling (though still using the non-Markovian structure over the chosen subsequence), reintroducing stochasticity.

From a theoretical perspective, the deterministic DDIM ($\eta=0$) process can be interpreted as approximating the solution trajectory of a specific probability flow Ordinary Differential Equation (ODE) related to the diffusion process. This connection bridges diffusion models with continuous-time generative models and provides a foundation for developing even more advanced ODE-based samplers, which we will explore in Chapter 6.

Understanding the DDIM sampling mechanism, its deterministic variant, and the ability to accelerate generation by skipping steps is fundamental. It not only provides a practical method for faster sampling with existing DDPM-trained models but also serves as a building block for many subsequent advancements in diffusion model sampling and distillation techniques covered later in this course.