The Bayesian Learning Rule Beyond KL Geometry

SOPHIA SKLAVIADIS, Wu Lin, Mario Figueiredo, André Martins and 2 more

Paris Poster Session 1, Wed, Dec 9, 12:30 PM–2:30 PM, Paris Poster Hall · Published 2026

Sparse Attention as Compact Kernel Regression

Sparse attention corresponds to compact kernel regression, with normalized ReLU and sparsemax arising from Epanechnikov kernels and α-entmax mapping to biweight and triweight kernels. This unifies sparsity with kernel design and yields competitive kernel-based transformers on language modeling and i

Saul Santos, Nuno Gonçalves, Daniel McNamee, Marcos Treviso and 1 more

Paris Poster Session 4, Thu, Dec 10, 5:30 PM–7:30 PM, Paris Poster Hall · Published 2026

DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention

DashAttention uses adaptive α-entmax to select variable key-value blocks per query, enabling fully differentiable hierarchical sparse attention that matches full-attention accuracy at 75% sparsity with faster inference than FlashAttention-3.

Yuxiang Huang, Nuno Gonçalves, Federico Alvetreti, Lei Li and 4 more

Paris Poster Session 6, Fri, Dec 11, 2:30 PM–4:30 PM, Paris Poster Hall · Published 2026