MAPLE: Many-Shot Adaptive Pseudo-Labeling for In-Context Learning
MAPLE uses influence-based pseudo-labeling to adaptively select many-shot ICL demonstrations, boosting LLM performance without extensive labeling costs.
Published May 22, 2025arXiv ↗
Only vote on papers you've read. Sign in with GitHub to vote.
MAPLE's adaptive pseudo-labeling delivers compelling many-shot ICL gains from unlabeled data without heavy labeling costs, though it remains unclear whether influence selection outperforms retrieval, pseudo-label errors are quantified, or computation overhead erodes savings under small pools…
Abstract
In-Context Learning (ICL) empowers Large Language Models (LLMs) to tackle diverse tasks by incorporating multiple input-output examples, known as demonstrations, into the input of LLMs. More recently, advancements in the expanded context windows of LLMs have led to many-shot ICL, which uses hundreds of demonstrations and outperforms few-shot ICL, which relies on fewer examples. However, this approach is often hindered by the high cost of obtaining large amounts of labeled data. To address this challenge, we propose Many-Shot Adaptive Pseudo-LabEling, namely MAPLE, a novel influence-based many-shot ICL framework that utilizes pseudo-labeled samples to compensate for the lack of label information. We first identify a subset of impactful unlabeled samples and perform pseudo-labeling on them by querying LLMs. These pseudo-labeled samples are then adaptively selected and tailored to each test query as input to improve the performance of many-shot ICL, without significant labeling costs. Extensive experiments on real-world datasets demonstrate the effectiveness of our framework, showcasing its ability to enhance LLM adaptability and performance with limited labeled data.