Efficient Test-Time Scaling via Self-Calibration
Self-Calibration distills self-consistency confidence into LLMs for reliable single-pass estimation, enabling confidence-based early stopping that improves MathQA accuracy to 83.6 with 16 samples.
Published Feb 25, 20251 citation▲ 15 on Hugging FaceCode ★ 22arXiv ↗
Only vote on papers you've read. Sign in with GitHub to vote.
Self-Calibration cleverly distills self-consistency into one pass for efficient early stopping, but unverified reliability, narrow benchmarks, and missing cost and calibration metrics leave its scaling claims incomplete.
Abstract
Increasing test-time computation is a straightforward approach to enhancing the quality of responses in Large Language Models (LLMs). While Best-of-N sampling and Self-Consistency with majority voting are simple and effective, they require a fixed number of sampling responses for each query, regardless of its complexity. This could result in wasted computation for simpler questions and insufficient exploration for more challenging ones. In this work, we argue that model confidence of responses can be used for improving the efficiency of test-time scaling. Unfortunately, LLMs are known to be overconfident and provide unreliable confidence estimation. To address this limitation, we introduce Self-Calibration by distilling Self-Consistency-derived confidence into the model itself. This enables reliable confidence estimation at test time with one forward pass. We then design confidence-based efficient test-time scaling methods to handle queries of various difficulty, such as Early-Stopping for Best-of-N and Self-Consistency with calibrated confidence. Experiments on three LLMs across six datasets demonstrate the effectiveness of our approach. Specifically, applying confidence-based Early Stopping to Best-of-N improves MathQA accuracy from 81.0 to 83.6 with a sample budget of 16 responses, indicating the efficacy of confidence-based sampling strategy at inference time.