Good Papers

Showing Efficient & distributed training Show all papers

74%Highly rated
?Highly ratedVote to see the score
arXivMetaU WarsawEfficient & distributed training

PyTorch Distributed: Experiences on Accelerating Data Parallel Training

PyTorch's distributed data parallel module uses gradient bucketing, communication-computation overlap, and synchronization skipping to achieve near-linear scalability on 256 GPUs.

Li Shen, Yanli Zhao, Rohan Varma, Omkar Salpekar and 7 more

Published Jun 28, 2020 · 111 citations · ▲ 13 on Hugging Face · Code ★ 103,810

– ReadersNo votes yet
9/21 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 9 of 21 reviewers recommend it
lenient 4/5
medium 3/11
strict 2/5
45%Niche pick
?Niche pickVote to see the score
NeurIPS 2026U Illinois at Urbana-ChampaignU ArizonaPurdue, PurdueEfficient & distributed training

Optimizing Retraining Schedules via Learning Curves

Jin Sima, Changlong Wu, Ananth Grama, Wojciech Szpankowski

Atlanta Poster Session 4, Thu, Dec 10, 4:30 PM–7:30 PM, Hall C1 · Published 2026

– ReadersNo votes yet
0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it
lenient 0/5
medium 0/10
strict 0/5
45%Niche pick
?Niche pickVote to see the score
NeurIPS 2026Nanyang TechnologicalFudanThe Hong Kong University of ScieEfficient & distributed training

Advanced Routing as Regularization Allocation for Efficient Diffusion Transformer Training

Qin MA, XIAOQI SUN, bo li, Yuquan Zhou and 1 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet
0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it
lenient 0/5
medium 0/10
strict 0/5
45%Niche pick
?Niche pickVote to see the score
NeurIPS 2026KTH Royal Institute of TechnologKTH - Royal Institute of TechnolEfficient & distributed training

LITE: A Lightweight Lazy Sampler for Efficient SGD

Amir Daghestani, Mikael Johansson

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet
0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it
lenient 0/5
medium 0/10
strict 0/5
57%Worth a look
?Worth a lookVote to see the score
NeurIPS 2026StanfordStanford University / Virtue AIEfficient & distributed training

Is Backpropagation Optimal? When Synthetic Gradients Improve Sample Efficiency

Yibo Jacky Zhang, Zeyu Tang, Sanmi Koyejo

Atlanta Poster Session 3, Thu, Dec 10, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet
1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 1 of 20 reviewers recommend it
lenient 0/5
medium 1/10
strict 0/5
45%Niche pick
?Niche pickVote to see the score
NeurIPS 2026Advanced Micro DevicesNorthwesternAMDEfficient & distributed training

Beyond the Trial-and-Error Loop: Hybrid Projection and Automated Tuning for Distributed Training

Anshu Raina, Peyman Razaghi, Yuankai Chen, Cheng Yao and 6 more

Atlanta Poster Session 3, Thu, Dec 10, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet
0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it
lenient 0/5
medium 0/10
strict 0/5
57%Worth a look
?Worth a lookVote to see the score
NeurIPS 2026U PisaEfficient & distributed training

Sustainability in the Loop: AI Model Development Should Be Multi-Objective

Matteo Mugnai, Francesco Pistolesi

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet
1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 1 of 20 reviewers recommend it
lenient 1/5
medium 0/10
strict 0/5
45%Niche pick
?Niche pickVote to see the score
NeurIPS 2026Czech Academy of SciencesCzech Academy of Sciences, InstiEfficient & distributed training

Match the Geometry, Skip the Surrogate: Extreme Low-Budget Optimization in High Dimensions

Michal Prusek, Adam Novozámský, Filip Sroubek

Paris Poster Session 4, Thu, Dec 10, 5:30 PM–7:30 PM, Paris Poster Hall · Published 2026

– ReadersNo votes yet
0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it
lenient 0/5
medium 0/10
strict 0/5
45%Niche pick
?Niche pickVote to see the score
NeurIPS 2026U OxfordEfficient & distributed training

Accelerating Neural Network Training with Augmented Koopman Dynamics

Jingyi Huang, Keyan Miao, Kostas Margellos, Paul Goulart

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet
0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it
lenient 0/5
medium 0/10
strict 0/5
45%Niche pick
?Niche pickVote to see the score
NeurIPS 2026NAVER AI LabNaver AI LabEfficient & distributed training

FOAM: Factored One-sided Adam-Moment for Practical and Scalable SOAP

Jaemyung Yu, Byeongho Heo, Sangdoo Yun, Dongyoon Han

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet
0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it
lenient 0/5
medium 0/10
strict 0/5
45%Niche pick
?Niche pickVote to see the score
NeurIPS 2026Norwegian University of Science Efficient & distributed training

Simplified Reversible Residual Networks

Erland B Olsson, Zhirong Yang

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet
0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it
lenient 0/5
medium 0/10
strict 0/5
57%Worth a look
?Worth a lookVote to see the score
NeurIPS 2026PurdueEfficient & distributed training

Talk Less, Work More: Communication-Efficient Decentralized Stochastic Approximation

Tianyu Cao, Haixiang Sun, Yang Xu

Atlanta Poster Session 5, Fri, Dec 11, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet
1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 1 of 20 reviewers recommend it
lenient 1/5
medium 0/10
strict 0/5
45%Niche pick
?Niche pickVote to see the score
NeurIPS 2026Sun Yat-sen University & SheSUN YAT-SEN UNIVERSITYThe University of SydneyNanjing University of Science anInformation Technology Service CEfficient & distributed training

Generalization Without Compression Penalty: A Stability Analysis of Error Feedback

Yifei Liang, Peng Wang, Yan Sun, Yingqi Liu and 3 more

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet
0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it
lenient 0/5
medium 0/10
strict 0/5
57%Worth a look
?Worth a lookVote to see the score
NeurIPS 2026Moscow Independent Research InstMoscow Institute of Physics and Efficient & distributed training

On Communication-Efficient Training of Ensembles in Federated Learning

Valery Parfenov, Mikhail Aleksandrov, Daniil Medyakov, Dmitry Bylinkin and 1 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet
1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 1 of 20 reviewers recommend it
lenient 1/5
medium 0/10
strict 0/5
45%Niche pick
?Niche pickVote to see the score
NeurIPS 2026BostonBoston University, BostonEfficient & distributed training

A Transformer-Derived Iterative Preconditioner

Patrick Lutz, Themistoklis Haris, Aditya Gangrade, Venkatesh Saligrama

Atlanta Poster Session 1, Wed, Dec 9, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet
0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it
lenient 0/5
medium 0/10
strict 0/5
45%Niche pick
?Niche pickVote to see the score
NeurIPS 2026University College London, UniveConcord AIStanfordEfficient & distributed training

An Analytical Model of Compute-limited Multistage Training Pipelines

Nishil Patel, Jin Hwa Lee, Basile Confavreux, Andrew Saxe

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet
0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it
lenient 0/5
medium 0/10
strict 0/5
45%Niche pick
?Niche pickVote to see the score
NeurIPS 2026Brookhaven National LabBrookhaven National LaboratoryEfficient & distributed training

Efficient Neural Field Learning via Adaptive Coverage and Focused Sampling

Guang Zhao, Xihaier Luo, Huan-Hsin Tseng, Seungjun Lee and 3 more

Atlanta Poster Session 5, Fri, Dec 11, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet
0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it
lenient 0/5
medium 0/10
strict 0/5
45%Niche pick
?Niche pickVote to see the score
NeurIPS 2026EPFLEPFL - EPF LausanneUmeåEfficient & distributed training

AcceleGrad#: Adaptive Geometry-Aware Acceleration

Hanka Goralija, Francesco Tonin, Kimon Antonakopoulos, Alp Yurtsever and 1 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet
0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it
lenient 0/5
medium 0/10
strict 0/5
70%Highly rated
?Highly ratedVote to see the score
NeurIPS 2026Massachusetts Institute of TechnEfficient & distributed training

Exact Instance Compression for Convex Empirical Risk Minimization via Color Refinement

A lossless color-refinement framework compresses convex empirical risk minimization instances exactly, accelerating linear, logistic, and kernel regression solvers.

Bryan Zhu, Ziang Chen

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet
4/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 4 of 20 reviewers recommend it
lenient 2/5
medium 2/10
strict 0/5
86%Must read
?Must readVote to see the score
NeurIPS 2026Michigan StateU MinnesotaU California, Santa BarbaraAlibaba Group USMichigan State University & Efficient & distributed training

Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization

Subspace gradient orthogonalization unifies low-rank projection with spectral optimization into ZO-Muon, cutting zeroth-order queries by 75% versus MeZO while boosting accuracy on LLM and vision fine-tuning.

Yicheng Lang, Changsheng Wang, Yihua Zhang, Mingyi Hong and 3 more

Atlanta Poster Session 3, Thu, Dec 10, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet
14/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 14 of 20 reviewers recommend it
lenient 4/5
medium 8/10
strict 2/5
Show 20 more papers