Good Papers: read the good ones

A stopgrad regression principle characterizes stationary points of stopgrad objectives and proves convergence to true flow maps while halving training memory.

Mark Goldstein, Max Shen, Zichu Wang, Aahlad Manas Puli and 1 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

11/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

83%Must read

?Must readVote to see the score

NeurIPS 2026Georgia Institute of Technology Deep learning theory

SGD Provably Prioritizes a Shortcut Spurious Feature in the XOR Model

SGD learns linear spurious correlations exponentially faster than XOR signals in two-layer ReLU networks, with dynamics that suppress true feature learning.

Tyler LaBonte, Vidya Muthukumar

Atlanta Poster Session 3, Thu, Dec 10, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet

13/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 13 of 20 reviewers recommend it

lenient 4/5

medium 6/10

strict 3/5

80%Must read

?Must readVote to see the score

NeurIPS 2026U Central Florida Deep learning theory

Bifurcation Models: Learning Set-Valued Solution Maps with Weight-Tied Dynamics

Bifurcation models learn set-valued solution maps via weight-tied dynamics, representing multi-branch attractor landscapes with almost everywhere regular selectors and outperforming single-branch supervision, though diversity requires explicit encouragement.

Caleb Jore, Jialin Liu

Atlanta Poster Session 2, Wed, Dec 9, 4:30 PM–7:30 PM, Hall C1 · Published 2026

– ReadersNo votes yet

12/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

71%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Imperial College London Universität St. Gallen Deep learning theory

Random Neural Network Expressivity for Non-Linear Partial Differential Equations

Random neural networks approximate time-dependent Sobolev functions with dimension-free rate 1/2 and efficiently solve nonlinear porous medium and compressible Navier-Stokes equations.

Muhammed Ali Mehmood, Lukas Gonon

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

6/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 6 of 20 reviewers recommend it

lenient 3/5

medium 2/10

strict 1/5

70%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Stony Brook Deep learning theory

Subcritical Signal Propagation at Initialization in Normalization-Free Transformers

Average partial Jacobian norms in transformers reveal subcritical signal growth in normalization-free architectures via tanh-like nonlinearities, explaining initialization sensitivity in DyT and Derf models.

Sergey Alekseev

Paris Poster Session 1, Wed, Dec 9, 12:30 PM–2:30 PM, Paris Poster Hall · Published 2026

– ReadersNo votes yet

5/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 5 of 20 reviewers recommend it

lenient 2/5

medium 3/10

strict 0/5

72%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026SpotlightCopenhagen U Copenhagen U Technology Nuremberg Worcester Polytechnic Institute Technion - Israel institue of Te Deep learning theory

On the Depth of Monotone ReLU Neural Networks and ICNNs

Monotone ReLU networks cannot compute or approximate maximum, ICNNs need depth n for it, and depth-k ICNNs cannot simulate some depth-2 ReLU networks.

Egor Bakaev, Florestan Brunck, Christoph Hertrich, Daniel Reichman and 1 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

8/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

72%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026SpotlightU Pennsylvania Sakana AI Northeastern Harvard Deep learning theory

Consistent Geometric Deep Learning via Hilbert Bundles and Cellular Sheaves

A Hilbert bundle convolutional framework defines HilbNets for infinite-dimensional manifold signals, proving discrete versions converge to continuous architectures and transfer across samplings.

Kartik Tandon, Julian J Gould, Tanishq Bhatia, Francesca Dominici and 2 more

Paris Poster Session 4, Thu, Dec 10, 5:30 PM–7:30 PM, Paris Poster Hall · Published 2026

– ReadersNo votes yet

8/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 8 of 20 reviewers recommend it

lenient 3/5

medium 3/10

strict 2/5

76%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026MIT EPFL Princeton EPFL - EPF Lausanne Deep learning theory

Provable Quantization with Randomized Hadamard Transform

Dithered randomized Hadamard quantization is unbiased and achieves mean squared error asymptotically matching dense random rotations at O(d log d) cost.

Ying Feng, Piotr Indyk, Michael Kapralov, Dmitrii Krachun and 1 more

Atlanta Poster Session 6, Fri, Dec 11, 4:30 PM–7:30 PM, Hall C1 · Published 2026

– ReadersNo votes yet

10/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

70%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Georgia Institute of Technology U Washington U Illinois at Urbana-Champaign Deep learning theory

Global Convergence of Four-Layer Matrix Factorization under Random Initialization

Gradient descent globally converges for randomly initialized four-layer matrix factorization with balanced regularization, avoiding saddles in polynomial time.

Minrui Luo, Weihang Xu, Xiang Gao, Maryam Fazel and 1 more

Atlanta Poster Session 1, Wed, Dec 9, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet

5/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

76%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Yale Deep learning theory

Decoupled Descent: Exact Test Error Tracking Via Approximate Message Passing

Decoupled descent cancels data-reuse biases via approximate message passing so training error tracks test error, enabling zero-cost validation and shrinking the generalization gap versus gradient descent.

Max Lovig

Atlanta Poster Session 3, Thu, Dec 10, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet

10/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

78%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026National University of Singapore Deep learning theory

InfoFlow: A Framework for Multi-Layer Transformer Analysis

InfoFlow proves multi-layer Transformers exponentially beat single-layer ones on retrieval tasks and tracks information propagation to explain multi-layer approximation efficiency.

Penghao Yu, Haotian Jiang, Zeyu Bao, Qianxiao Li

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

11/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

72%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026OralTechnion Technion - Israeli institute of Deep learning theory

Learning Reveals Invisible Structure in Low-Rank RNNs

Deriving reduced ODEs for low-rank RNN learning reveals loss-invisible overlaps that encode training history and expose hidden connectivity differences.

Yoav Ger, Omri Barak

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

8/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

78%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026U Oxford Boston Deep learning theory

Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit

Regularized Newton training of overparameterized neural networks converges to a deterministic NNTK limit with exponentially fast uniform convergence across all frequencies, avoiding gradient descent's spectral bias.

Konstantin Riedl, Justin Sirignano, Konstantinos Spiliopoulos

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

11/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

72%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026RIKEN Imperial College London The University of Tokyo / RIKEN Deep learning theory

Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformers

Training a linear FFN in mean-field transformers drives token distributions to escape attention-induced clustering near final layers.

Noboru Isobe, Daisuke Inoue, Masaaki Imaizumi

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

8/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 8 of 20 reviewers recommend it

lenient 2/5

medium 4/10

strict 2/5

72%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Friedrich-Alexander Universität U Wuerzburg U Oxford Technical University of Munich Deep learning theory

Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regime

Mean-field transformers exhibit rapid token distribution concentration onto projection-driven limits with explicit Wasserstein bounds scaling in inverse temperature β and time t.

Albert Alcalde, Leon Bungert, Konstantin Riedl, Tim Roith

Paris Poster Session 4, Thu, Dec 10, 5:30 PM–7:30 PM, Paris Poster Hall · Published 2026

– ReadersNo votes yet

8/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

76%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Gatsby Computational Neuroscienc University College London, Unive IST Austria Stanford Deep learning theory

Optimal Representation Size: High-Dimensional Analysis of Pretraining and Linear Probing

High-dimensional analysis of pretraining via PCA and linear probing derives exact errors versus representation size, showing compression helps with abundant unlabeled but scarce labeled data.

Valentina Njaradi, Clémentine Dominé, Rachel A Swanson, Marco Mondelli and 1 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

10/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

72%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Columbia Deep learning theory

Fixed Universal Transformers

Fixed universal transformers simulate any target transformer via input embeddings with frozen internal parameters, and random initialization achieves universality almost surely.

Jingwen Liu, Alexandr Andoni, Daniel Hsu

Atlanta Poster Session 3, Thu, Dec 10, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet

8/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 8 of 20 reviewers recommend it

lenient 3/5

medium 4/10

strict 1/5

67%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Champalimaud Foundation Champalimaud Centre for the Unkn Deep learning theory

Universal Approximation Theorems for Dynamical Systems with Infinite-Time Horizon Guarantees

Framework guarantees universal approximation for multistable dynamics with infinite-time horizon guarantees, linking topological properties to training metrics.

Ábel Ságodi, Memming Park

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

2/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 2 of 20 reviewers recommend it

lenient 0/5

medium 2/10

strict 0/5

45%Niche pick

?Niche pickVote to see the score

NeurIPS 2026U Tübingen & IMPRS-IS Allen Institute, Neural Dynamics Stanford U Alberta Deep learning theory

Parameter symmetries determine representational geometry in overparameterized nonlinear networks

The poster argues parameter symmetries determine representational geometry in overparameterized nonlinear networks.

Marvin Theiss, Lukas Braun, Andrew Saxe, Erin Grant

Paris Poster Session 3, Thu, Dec 10, 12:30 PM–2:30 PM, Paris Poster Hall · Published 2026

– ReadersNo votes yet

0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it

lenient 0/5

medium 0/10

strict 0/5

71%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026CNRS and ENS Deep learning theory

Muon Dynamics as a Spectral Wasserstein Flow

Spectral Wasserstein distances unify normalized matrix flows, proving Muon dynamics are gradient flows with Benamou-Brenier equivalence for monotone norms.

Gabriel Peyré

Paris Poster Session 4, Thu, Dec 10, 5:30 PM–7:30 PM, Paris Poster Hall · Published 2026

– ReadersNo votes yet

6/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

74%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Sissa SISSA, Trieste, Italy INRIA Deep learning theory