BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CAIL//Events//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:CAIL Events
BEGIN:VEVENT
UID:2025-10-03-jingfeng-wu@cail.columbia.edu
DTSTAMP:20260823T060917Z
DTSTART:20251003T150000Z
DTEND:20251003T160000Z
SUMMARY:ML Seminar: Jingfeng Wu - Gradient Descent Dominates Ridge: A Stat
 istical View on Implicit Regularization
LOCATION:School of Social Work\, Room C03
DESCRIPTION:A key puzzle in deep learning is how simple gradient methods f
 ind generalizable solutions without explicit regularization. This talk dis
 cusses the implicit regularization of gradient descent (GD) through the le
 ns of statistical dominance. Using least squares as a clean proxy\, we pre
 sent two surprising findings.\n\nFirst\, GD dominates ridge regression: wi
 th comparable regularization\, the excess risk of GD is always within a co
 nstant factor of ridge\, but ridge can be polynomially worse even when tun
 ed optimally. Second\, GD is incomparable with SGD. While it is known that
  for certain problems GD can be polynomially better than SGD\, the reverse
  is also true: we construct problems\, inspired by benign overfitting theo
 ry\, where optimally stopped GD is polynomially worse. Finally\, GD domina
 tes SGD for a significant subclass of problems — those with fast and con
 tinuously decaying covariance spectra — which includes all problems sati
 sfying the standard capacity condition.\n\nThis is joint work with Peter B
 artlett\, Sham Kakade\, Jason Lee\, and Bin Yu.\n\nhttps://cail.columbia.e
 du/events/2025-10-03-jingfeng-wu
URL:https://cail.columbia.edu/events/2025-10-03-jingfeng-wu
END:VEVENT
END:VCALENDAR