CS Events

Faculty Candidate Talk

When Second Order Methods Shine: Big Batches, Bayes, and Bilevel


Download as iCal file

Monday, February 28, 2022, 10:30am - 12:00pm


Scheduled Zoom meeting.

Topic: Guodong Zhang talk
Time: Feb 28, 2022 10:30 AM Eastern Time (US and Canada)

Join Zoom Meeting

Join by SIP

Meeting ID: 928 7559 4180
Password: 449292
One tap mobile
+16465588656,,92875594180# US (New York)
+13017158592,,92875594180# US (Washington DC)

Join By Phone
+1 646 558 8656 US (New York)
+1 301 715 8592 US (Washington DC)
+1 312 626 6799 US (Chicago)
+1 669 900 9128 US (San Jose)
+1 253 215 8782 US (Tacoma)
+1 346 248 7799 US (Houston)
Meeting ID: 928 7559 4180
Find your local number: https://rutgers.zoom.us/u/ab1lVp5Bch

Join by Skype for Business

If you have any questions, please <a href="https://it.rutgers.edu/help-support/">contact the Office of Information Technology Help Desk</a>

Speaker: Guodong Zhang


Guodong Zhang is a PhD candidate in the machine learning group at the University of Toronto, advised by Roger Grosse. His research lies at the intersection between machine learning, optimization, and Bayesian statistics. In particular, his research focuses on understanding and improving algorithms for optimization, Bayesian inference, and multi-agent games in the context of deep learning. He has been recognized through the Apple PhD fellowship, Borealis AI fellowship, and many other scholarships. In the past, he has also spent time at Institute for Advanced Study of Princeton and industry research labs (including DeepMind, Google Brain, and Microsoft Research).

Location : Via Zoom

Event Type: Faculty Candidate Talk

Abstract: Many challenges in modern machine learning involve the three fundamental problems of optimization, Bayesian inference, and multi-player games. In this talk, I will discuss how the use of second-order information – e.g., curvature or covariance – can help in all three problems, yet with vastly different roles in each. First, I will present a noisy quadratic model, which qualitatively predicts scaling properties of a variety of optimizers and in particular suggests that second-order optimization algorithms would extend perfect scaling to much bigger batches. Second, I will show how we can derive and implement scalable and flexible Bayesian inference algorithms from standard second-order optimization algorithms. Third, I will describe a novel second-order algorithm that finds desired equilibria and saves us from converging to spurious fixed points in two-player sequential games (i.e. bilevel optimization) or even more general settings. Finally, I will conclude how my research would pave the way towards intelligent machines that can learn from experience efficiently, reason about their own decisions, and act in our interests.


Rutgers University School of Arts and Sciences

Contact  Ahmed Elgammal