Our next InfraAI Lecture will feature Vishal Misra, Columbia University, speaking on "From Prediction to Computation: Bayesian Geometry, Transformers, and Mathematical Discovery
Date: Friday, November 13, 2026
Time: 2:00 - 3:00 PM
Location: Gates-Dell Complex Auditorium (GDC 2.216)
Zoom link available through registration.
Abstract: Large language models are trained with a remarkably simple objective: predict the next token. Yet they exhibit capabilities that seem to go far beyond prediction, including reasoning, in-context learning, and mathematical discovery. What computations do transformers actually learn, and how does the training objective give rise to them?
In this talk, I will describe a series of results investigating the Bayesian structure of transformer learning. Using controlled experiments where the optimal Bayesian predictor is known analytically, we examine how transformers learn to perform inference, how cross-entropy gradients shape attention geometry, and whether these geometric signatures persist in large language models. We also explore the limits of these representations, including a striking result where transformers can identify an underlying program but fail to reuse it reliably.
Finally, I will connect these findings to AI-assisted mathematical discovery, examining the distinction between verifying a proof, finding one, and deciding what is worth proving. This leads to a broader systems perspective: what an AI system can accomplish depends not just on what the model has learned, but on its architecture and the feedback loops surrounding it.
About the Speaker: Vishal Misra is the RKS Family Professor of Computer Science and Electrical Engineering and Vice Dean of Computing and AI at Columbia Engineering. A Fellow of the ACM and IEEE, his research spans networking and the mathematical foundations of large language models. As a graduate student in the early 1990s he co-founded Cricinfo, and today he is a minority owner and Dean of Cricket Analytics of Major League Cricket’s San Francisco Unicorns. His net neutrality work shaped India’s regulatory framework, including testimony before the Indian Parliament. He holds a BTech from IIT Bombay, where he is a Distinguished Alumnus, and a PhD from UMass Amherst where he is a Distinguished Young Alumnus.
