Learning to Learn a Language

Prior-Fitted Language Model, trained solely on synthetic non-linguistic data, learns to infer and predict real languages from context with frozen weights, achieving strong cross-lingual compression and reasoning without ever seeing real text.

Lennart Carstens-Behrens, Holger Fröhlich

Published Oct 5, 2026 · ▲ 7 on Hugging Face · Code ★ 1

Labels Override Definitions in Jev-Style Typed Decision Models

Typed decision models exhibit option-label bias because prompts prepend labels to definitions, letting label semantics override rules; removing labels or altering formatting fixes it.

Seyedarmin Azizi, Erfan Baghaei Potraghloo, Massoud Pedram

Published Oct 1, 2026 · 0 citations · ▲ 5 on Hugging Face

Efficient Task Adaptation in Large Language Models: A Survey of Weight-Based, Prompt-Based, and Embedding-Based Adaptations

This survey unifies weight, prompt, and embedding adaptation methods for large language models into one taxonomy, analyzing trade-offs and cross-paradigm relationships.

Jungwon Park, Changin Choi, Jimyeong Kim, Nojun Kwak and 1 more

Published Oct 1, 2026 · 0 citations

Capturing In-Context Learning Dynamics with Task Operators

Task Operator captures ICL as stable per-task affine attention transformations, enabling efficient zero-shot replay that nearly matches in-context performance and scales beyond context limits.

Guangzhi Xiong, Zhenghao He, Bohan Liu, Sanchit Sinha and 2 more

Atlanta Poster Session 5, Fri, Dec 11, 10:00 AM–1:00 PM, Hall C1 · Published Oct 1, 2026 · 0 citations

BDH-CQ: In-Context Learning with Recurrent Latent Reasoning

BDH-CQ combines in-context learning with recurrent latent reasoning, achieving 29.5% ARC-AGI-1 pass@2 at $0.0007 per task to set a new cost-efficiency frontier.

Björn Engdahl, Adrian Kosowski, Jan Chorowski, Zuzanna Stamirowska and 5 more

Published Aug 10, 2026 · 0 citations · ▲ 797 on Hugging Face · Code ★ 11,072

TextReg: Mitigating Prompt Distributional Overfitting via Regularized Text-Space Optimization

TextReg mitigates prompt distributional overfitting via regularized text-space optimization, improving out-of-distribution accuracy by up to 16.5% over prior methods.

傅卢成, Ye Yu, Yiyang Wang, Yiqiao Jin and 3 more

Published May 20, 2026 · 0 citations · ▲ 7 on Hugging Face · Code

MAPLE: Many-Shot Adaptive Pseudo-Labeling for In-Context Learning

MAPLE uses influence-based pseudo-labeling to adaptively select many-shot ICL demonstrations, boosting LLM performance without extensive labeling costs.

Zihan Chen, Song Wang, Zhen Tan, Jundong Li and 1 more

Published May 22, 2025 · 0 citations

Divide, Reweight, and Conquer: A Logit Arithmetic Approach for In-Context Learning

LARA improves in-context learning by dividing long demonstrations into shorter parallel groups and reweighting their logits via non-gradient optimization, boosting accuracy and memory efficiency over baselines on BBH and MMLU.

Chengsong Huang, Langlin Huang, Jiaxin Huang

Published Oct 14, 2024 · 1 citation · ▲ 1 on Hugging Face · Code ★ 8

ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT

ChatIE reframes zero-shot information extraction as multi-turn ChatGPT dialogue, surpassing some fully supervised models on several benchmark datasets.

Wei, Xiang, Xingyu Cui, Ning Cheng, Xiaobin Wang and 8 more

Published Feb 20, 2023 · 147 citations

CAPO: A Primal-Dual Framework for Constraint-Aware Prompt Optimization

Victor Ye Dong, Reid Pryzant, Yi Liu, Jian Jiao

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

SPLICE: Structured Prompt Local Iterative Combinatorial Evolution

Dr. Anish Acharya, Phillip Studans, Amit Dhanda, Ninad V Rao and 3 more

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Read-Only Zero-Shot Classifier Expansion from Pairwise Semantics

Zhiqiang Zhong, Jun Pang

Paris Poster Session 2, Wed, Dec 9, 5:00 PM–7:00 PM, Paris Poster Hall · Published 2026

Reinforced Fast Weights via Next-Sequence Prediction

Hee Seung Hwang, Xindi Wu, Sanghyuk Chun, Zhiwei Deng and 1 more

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Transformers Can Learn Multiclass Classification In-Context: Isotropy Governs Generalization

Daehan Yoon, Chulhee Yun

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

From Contexts to Conditionals: Statistical Self-Consistency of Persona Prompting

Patrik Wolf, Thomas Kleine Buening, Andreas Krause, Celestine Mendler-Dünner

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

One-Layer Transformers Provably Learn In-Context K-Nearest Neighbor Prediction with Chain-of-Thought

Lyumin Wu, Yuan Cao

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Optimal In-Context Learning of Autoregressive Processes under Heterogeneous Second-Order Moments of the Prompts

Hanna Tseran, Masaaki Imaizumi

Atlanta Poster Session 1, Wed, Dec 9, 10:00 AM–1:00 PM, Hall C1 · Published 2026

Salvation Lies Within: Proactive Prefix Re-forming for LLM-based Tagging

SHULAN WANG, YingJie Zhu, Yuting Yan, Ke Cheng and 7 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Prompt Optimization Makes Misalignment Legible

Caleb Biddulph, Micah Carroll

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Closed-Loop Alignment: Socially Coupled In-Context Learning and Relational Posterior Collapse

Hidenori Tanaka

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Knowing You before You Speak: User State Modeling for LLM-Based Personalized Dialogue

jiani luo, Xiaoyan Zhao, Yang Zhang, Shuyi Miao and 3 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

TriPrompt: Progressive Local Prompting for Few-shot Out-of-Distribution Detection

Ziyou Xiang, Chaowei Fang, Zhihong Wu, Jiliang Li and 2 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Logarithmic Depth Suffices for In-Context Gradient Descent

Yingze Li, Dong Wang, Xianglong Liu, Qingyun Zou and 3 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Prompt-Driven Exploration

Sunshine Jiang, John Marangola, David Zhang, Raghuram Kowdeed and 5 more

Atlanta Poster Session 3, Thu, Dec 10, 10:00 AM–1:00 PM, Hall C1 · Published 2026

A Unified Theoretical Framework for Task Recognition and Task Learning in In-Context Learning

Zhixuan Pan, Li Cao, Yiqi Dong, Chenyu Gan and 2 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Simplicity is Enough: ReAct Agents for Prompt Optimization

Andrei Rusu, Andrei Dumitrescu, Adrian C Badea, Cosmin Maria and 4 more

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Prefix-Tuning for Arbitrary Output Sequences on Pretrained Transformers

Peter Cho-Ho Lam, Ziyi Wang, Zirui Zhou

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

MoMHa: Multi-Objective Optimization of LLM Harnesses over Accuracy, Safety, and Tokens

MoMHa treats LLM harness design as multi-objective search over accuracy, safety, and token cost, outperforming baselines across 17 domains via joint-reward optimization.

Subhojyoti Mukherjee, Mehrab Tanjim

Atlanta Poster Session 6, Fri, Dec 11, 4:30 PM–7:30 PM, Hall C1 · Published 2026

Automated Reformulation of Robust Optimization via Memory-Augmented Large Language Models

AutoREM is a tuning-free memory-augmented framework that automates robust optimization reformulation via experience memory and improves accuracy across models.

Jinbiao Chen, Shuang Jin, Guoyun Zhang, Junyu Zhang and 2 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Learning to Follow In-Context Watermark Instructions via Self-Distillation

ICWBench reveals current LLMs fail at in-context watermarking, and self-distillation with reinforcement learning raises watermark detectability near perfect while preserving quality.

Yepeng Liu, Tianyi Chen, Xuandong Zhao, Dawn Song and 1 more

Atlanta Poster Session 2, Wed, Dec 9, 4:30 PM–7:30 PM, Hall C1 · Published 2026

LLMs can construct powerful representations and streamline sample-efficient supervised learning

LLMs generate global and local rubrics to standardize multimodal inputs, significantly outperforming clinical baselines on 15 EHRSHOT tasks via sample-efficient supervised learning.

Ilker Demirel, Lawrence Shi, Zeshan Hussain, David Sontag

Atlanta Poster Session 3, Thu, Dec 10, 10:00 AM–1:00 PM, Hall C1 · Published 2026

Prompts to Proxies: Emulating Human Preferences via a Compact LLM Ensemble

P2P uses adaptive prompting and L1-regularized regression to build compact LLM ensembles that emulate human preferences at low cost without fine-tuning. It achieves 0.014 test MSE on American Trends Panel surveys for about $0.80 each and outperforms supervised baselines with under 3% of their traini

Bingchen Wang, Zi-Yu Khoo, Jingtan Wang

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning

Task vector design via distributional alignment with in-context learning minimizes next-token probability discrepancy, yielding a linear method that improves accuracy by 9.2% and enables cross-scale transfer.

Jihoon Kwon, Jiwon Choi, Jy-yong Sohn

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Memory Inception: Latent-Space KV Cache Manipulation for Steering LLMs

Memory Inception steers LLMs by inserting text-derived KV banks at selected layers, improving control with up to 118× less storage than prompting.

Zeyi (Andy) Liu, Michael Zhang, Ilana Greenberg, Adam Alnasser and 2 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems

AgentGrad improves multi-agent prompt optimization via sequential intervention and semantic gradient clustering, achieving state-of-the-art performance with 2.5x faster optimization and 21.8% lower cost.

Jaewon Chu, jinwoo seo, Jaewon Cho, Jeehye Na and 3 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026 · ▲ 107 on Hugging Face

Words That Make Language Models Perceive

Sensory prompting cues text-only LLMs to activate vision- or audio-aligned representations, aligning them with specialist encoders without multimodal training.

Sophie L. Wang, Phillip Isola, Brian Cheung

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026 · ▲ 4 on Hugging Face · Code ★ 42

SePO: Self-Evolving Prompt Agent for System Prompt Optimization

SePO evolves its own prompt agent's system prompt via self-referential open-ended search to optimize task agents, outperforming baselines by 4.49 points across five benchmarks.

Wangcheng Tao, Han Wu, Weng-Fai Wong

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026 · ▲ 7 on Hugging Face

PULSE: Identifying Demonstration-Utility Features with Sparse Autoencoders

PULSE uses sparse autoencoders to identify internal features linked to demonstration utility and improves selection across classification, generation, and reasoning tasks.

Chenduo Hao, Chuanbao Gao, Pinjun Zeng, Jingze Zhu and 3 more

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026