Multiplicative Reweighting for Robust Neural Network Optimization
Multiplicative reweighting improves neural network optimization under label noise by updating example weights via multiplicative updates, boosting accuracy on CIFAR and Clothing1M while enhancing adversarial robustness.
Published Oct 13, 20251 citationPaper ↗
Only vote on papers you've read. Sign in to vote.
Abstract
Abstract. Neural networks are widespread due to their powerful performance. Yet they degrade in the presence of noisy labels at training time. Inspired by the setting of learning with expert advice, where multiplicative weight (MW) updates were recently shown to be robust to moderate data corruptions in expert advice, we propose to use MW for reweighting examples during neural network optimization. We theoretically establish the convergence of our method when used with gradient descent and prove its advantages in one-dimensional cases. We then validate empirically our findings for the general case by showing that MW improves the accuracy of neural networks in the presence of label noise on CIFAR-10, CIFAR-100, and Clothing1M. We also show the impact of our approach on adversarial robustness.