Empirical Variational Autoencoder
Empirical Variational Autoencoder learns autoregressive latent priors empirically via one linear layer to close the VAE prior-posterior gap, yielding high-fidelity sequential generation competitive with diffusion models at much faster inference.
Published Oct 5, 2026▲ 7 on Hugging FaceCode ★ 4arXiv ↗
Only vote on papers you've read. Sign in with GitHub to vote.
EVA's single linear layer closes the VAE latent gap for fast sequential generation, but critics question whether the empirical prior memorizes training statistics, scales to long sequences, and earns its speed claims against diffusion with rigorous…
Abstract
We present Empirical Variational Autoencoder, a general generative framework for continuous-valued (i.e., non-vector-quantized) sequences. EVA is based on the evidence lower bound of the Variational Autoencoder (VAE) but learns autoregressive latent priors empirically from training data, which can be implemented only by an additional single linear layer on top of VAEs. By replacing the conventional standard-Gaussian constraint with the self-predicted priors, EVA significantly alleviates the latent distribution gap between prior and posterior which is typically observed in conventional VAEs, and leads to high-fidelity ancestral sampling for sequential data generation. Extensive experiments on image and sound synthesis demonstrate that EVA achieves competitive generation quality with autoregressive diffusion baselines despite its much faster inference time.