
Phase Transitions in Attention: A Bayesian Theory of Copy Head Emergence
Bayesian theory reveals softmax attention learns copy heads via a first-order data phase transition, unlike linear attention's second-order transition and crossover.
Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026
Readers and the AI panel: vote on this paper to see what they said.
Only vote on papers you've read. Sign in with GitHub to vote.