BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CAIL//Events//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:CAIL Events
BEGIN:VEVENT
UID:2025-11-21-andrej-risteski@cail.columbia.edu
DTSTAMP:20260823T060936Z
DTSTART:20251121T160000Z
DTEND:20251121T170000Z
SUMMARY:ML Seminar: Andrej Risteski - Architectural Choices in Scientific 
 ML: A View Through the Lens of Theory
LOCATION:School of Social Work\, Room C03
DESCRIPTION:In deep learning\, small architectural changes—such as resid
 ual connections or normalization layers—have often had outsized impact. 
 This talk examines how similar effects arise in recent applications of dee
 p learning to the sciences. The central theme is that the architectural ch
 anges we identify are not suggested by current benchmarks\, which remain m
 uch less mature than they are in image or language domains. Instead\, they
  become visible through the right theoretical lenses. We will showcase sev
 eral vignettes spanning graph neural networks (GNNs)\, time-dependent part
 ial-differential equations (PDEs)\, and steady-state PDEs.\n\nThe first se
 tting concerns graphs with bottlenecks or hubs: augmenting GNNs with edge-
 level state yields (provable) gains under constraints on depth and memory.
  We establish this using techniques from time–space tradeoffs in theoret
 ical computer science\, and show that neither "symmetry-only" theoretical 
 accounts nor standard GNN benchmarks would detect this separation. The nex
 t setting concerns time-dependent PDEs\, where adding an explicit memory l
 ayer via state-space models (e.g. S4) has negligible effect under full obs
 ervability\, but substantial impact under partial observation. This kind o
 f phenomenon is predicted by Mori–Zwanzig theory—which also inspired t
 he architectural change. Finally\, in steady-state PDEs and operator learn
 ing\, we show that Deep Equilibrium Model (DEQ)-based architectural change
 s have efficiency and robustness benefits. Here\, the design is motivated 
 by representation-theoretic constructions that simulate "unrolled" gradien
 t descent in function space.\n\nhttps://cail.columbia.edu/events/2025-11-2
 1-andrej-risteski
URL:https://cail.columbia.edu/events/2025-11-21-andrej-risteski
END:VEVENT
END:VCALENDAR