Abstract:

Transformers have become a powerful tool in modern machine learning, but their behavior becomes increasingly difficult to understand as they grow larger and deeper. In this talk, I will use differential equations and continuum models to develop a mathematical view of transformers. The basic idea is to describe the collective behavior of many tokens, attention heads, and layers rather than tracking each component individually. This leads to a continuous model that captures both the forward computation and the training process. I will discusshow these models relate to finite transformers and what they can tell us about training and convergence.

The talk will illustrate how differential equations can provide a useful mathematical lens for understanding transformers and how they learn.

Bio: 

Hailiang Liu is an internationally recognized applied mathematician whose research bridges partial differential equations, numerical analysis, and the mathematical foundations of deep learning and artificial intelligence. He is Professor of Mathematics and Computer Science (by Courtesy) at Iowa State University and recently served as a Program Director in the Applied Mathematics Program at the National Science Foundation (2024–2026). He received his M.S. in from Tsinghua University in 1988 and Ph.D. from the Chinese Academy of Sciences in 1995. Following a Humboldt Research Fellowship and a postdoctoral position at UCLA, he joined Iowa State University in 2002. His current research focuses on applied analysis, numerical analysis, deep learning, and artificial intelligence.