All Events

Is Exploration on Hard Problems in LLM RL a Solved Problem?

Aviral Kumar · Carnegie Mellon University

Friday, October 16, 2026 · 11:00 AM

SSW Room 903

There has been remarkable progress in using LLMs to solve hard problems, even just over the past few months, driven by scaling both pretraining and post-training. Many of these problems involve a fundamental search or “exploration” bottleneck, as it is known in the reinforcement learning literature. But does recent progress mean that we have solved exploration? In other words, can we take a sufficiently powerful pretrained model, scale its test-time compute (TTC), and expect it to solve essentially any hard problem? In this talk, I will take a broader view of exploration in LLMs, give you a high-level view on some of our past work on this problem over the last 1 year, and discuss recent efforts from my group to push the limits of exploration using multi-agent systems and weight-update-based internalization. I will conclude with several open questions about what remains unsolved in my biased opinion and where progress will come from next.

AMSTERDAM AVENUEBROADWAYCLAREMONT AVENUECOLLEGE WALKMORNINGSIDE DRIVERIVERSIDE DRIVE116TH STREET118TH STREET119TH STREET120TH STREET121ST STREET122ND STREETCSBMuddNWCCEPSRSCHDodge HallKent HallRussellUris HallZankelLow LibraryAlma Mater1116 StMorningside ParkNSchool of Social Work
SSW Room 903 · School of Social Work Map data © OpenStreetMap contributors
morningside campus

School of Social Work

SSW Room 903

Map data © OpenStreetMap contributors