BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CAIL//Events//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:CAIL Events
BEGIN:VEVENT
UID:2029-05-22-phillip-isola@cail.columbia.edu
DTSTAMP:20260823T060937Z
DTSTART:20260522T180000Z
DTEND:20260522T190000Z
SUMMARY:Vision Seminar: Phillip Isola - Neural Thickets: Diverse Task Expe
 rts Are Dense Around Pretrained Weights
LOCATION:CSB 480
DESCRIPTION:Pretraining produces a learned parameter vector that is typica
 lly treated as a starting point for further iterative adaptation. In this 
 work\, we instead view the outcome of pretraining as a distribution over p
 arameter vectors\, whose support already contains task-specific experts. W
 e show that in small models such expert solutions occupy a negligible frac
 tion of the volume of this distribution\, making their discovery reliant o
 n structured optimization methods such as gradient descent. In contrast\, 
 in large\, well-pretrained models the density of task-experts increases dr
 amatically\, so that diverse\, task-improving specialists populate a subst
 antial fraction of the neighborhood around the pretrained weights. Motivat
 ed by this perspective\, we explore a simple\, fully parallel post-trainin
 g method that samples N parameter perturbations at random\, selects the to
 p K\, and ensembles predictions via majority vote. Despite its simplicity\
 , this approach is competitive with standard post-training methods such as
  PPO\, GRPO\, and ES for contemporary large-scale models.\n\nhttps://cail.
 columbia.edu/events/2029-05-22-phillip-isola
URL:https://cail.columbia.edu/events/2029-05-22-phillip-isola
END:VEVENT
END:VCALENDAR