83%Must read
?Must readVote to see the score
Bug or Feature$^2$: Weight Drift, Activation Sparsity, and Spikes
Standard losses and biased activations induce negative weight drift that drives early training dynamics and extreme sparsity across architectures, with squared activations sharply improving accuracy until a cliff near 70% sparsity unless clipping controls intermediate spikes.
Paris Poster Session 1, Wed, Dec 9, 12:30 PM–2:30 PM, Paris Poster Hall · Published 2026 · ▲ 1 on Hugging Face · Code ★ 1
– ReadersNo votes yet
13/20 AI panelreviewers recommend it
Readers and the AI panel: vote on this paper to see what they said.
Only vote on papers you've read. Sign in with GitHub to vote.
AI panel: 13 of 20 reviewers recommend it
lenient 4/5
medium 7/10
strict 2/5