Morteza Mardani at ECE Spring 2025 Colloquium

Steering diffusion models for next-gen AI

Diffusion models are revolutionizing generative AI with unmatched capabilities in representing complex data—images, videos, 3D structures, and scientific phenomena. This talk examines their transformative impact on AI’s future, focusing on NVIDIA's pioneering role. We explore plug-and-play methods for steering foundational diffusion models to solve complex downstream tasks, including guidance and score distillation sampling. We also discuss integrating reinforcement learning with diffusion models for reward-based training to develop even more powerful generative AI systems.