Retrieval-Centric Deep Learning in Growing Nonparametric Neural Networks

Replacing linear attention with RBF and softmax kernels yields principled retrieval-centric deep learning for growing neural networks that store key-value pairs per training point, improving image classification and linking to fixed-network optimizers via MesaNet and DeltaNet.

Maximilian Schlegel, Rajai Nasser, Seijin Kobayashi, Yanick Schimpf and 5 more

Published Oct 2, 2026 · 0 citations · ▲ 8 on Hugging Face · Code ★ 3