Probabilistic Inference for Controlling Diffusion Models

Diffusion models are probabilistic models, and controlling a diffusion model at inference time is a probabilistic inference problem. In this talk, we will take a tour through inference-time control for diffusion models and organize the landscape around two broad ideas: sequential control, where we guide and refine samples as generation proceeds, and parallel control, where multiple samples along denoising trajectories interact towards the target distribution. For each family, we will look at both approximate and exact methods: what they are doing probabilistically, where the approximations come from, and when exactness can be recovered. Together, these perspectives give us a simple roadmap for understanding the methods, their trade-offs, and when each style of control is most useful.

Speaker bio

Jiajun He (opens in new tab) is a PhD student in the Machine Learning Group at the University of Cambridge, and currently interning at Microsoft Research New England. He works broadly on probabilistic machine learning, with a particular interest in the interaction between path-space probabilistic methods and generative models. His research spans diffusion and flow-based models, nonequilibrium methods, sampling, and information theory.

Series: MSR New England Generative Modeling & Sampling Seminar