Good Papers

Showing papers from Gatsby Unit, University College London Show all papers

45%Niche pick
?Niche pickVote to see the score
NeurIPS 2026Imperial College LondonEPFLGatsby Unit, University College LLM pretraining & scaling laws

Why Routers Freeze: Infinite Width Learning Dynamics for Mixture of Experts

Anish Dhir, Volkan Cevher, Leena Chennuru Vankadara

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet
0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it
lenient 0/5
medium 0/10
strict 0/5
86%Must read
?Must readVote to see the score
NeurIPS 2026Gatsby Unit, University College AmazonUniversity College London, UniveU TübingenLLM pretraining & scaling laws

How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable Parameterization

Researchers derive maximally scale-stable parameterizations for Mixture-of-Experts via dynamical mean-field theory, yielding robust learning-rate transfer and monotonic scaling gains across regimes.

Leena Chennuru Vankadara, Moritz Haas, Luke Hayward, Sebastian Bordt and 1 more

Paris Poster Session 6, Fri, Dec 11, 2:30 PM–4:30 PM, Paris Poster Hall · Published 2026 · ▲ 1 on Hugging Face · Code ★ 4

– ReadersNo votes yet
14/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 14 of 20 reviewers recommend it
lenient 3/5
medium 8/10
strict 3/5