Latent-MOPD: Latent Multi-Teacher On-Policy Distillation

Latent-MOPD distills multi-teacher LLM specialists via hidden-state and prediction-level on-policy supervision, outperforming token-only and representation-only baselines across math, code, and logic benchmarks.

Zhengyu Fang, Seoyeon Hong, Jie Yang, Muyang Li and 3 more

Published Oct 1, 2026 · 0 citations · ▲ 63 on Hugging Face

Local Support Learning

Local Support Learning pairs weight adapters with GMM gating to keep updates local, resolving catastrophic forgetting in LLMs up to 7B parameters without prior data.

Assaf Ben-Kish, Akarsh Kumar, James Glass, Raja Giryes

Published Oct 1, 2026 · 0 citations · ▲ 25 on Hugging Face · Code ★ 13

LLM2Jev: LLMs Are Already Jev-Style Decision Models -- When and How to Fine-Tune Them

LLM2Jev extracts calibrated Jev-style decisions from LLM token probabilities via training-free inference or tree-factorized fine-tuning, showing strong 4B models already match specialized decision models while fine-tuning mainly helps weaker backbones and specific tasks without degrading generation.

Yinheng Li, Justin Wagle

Published Oct 1, 2026 · 0 citations · ▲ 6 on Hugging Face

FedFit: Federated Fine-Tuning of LLMs via Vector-Bank Parameterization and Quantization

FedFit reduces federated LLM fine-tuning overhead via vector-bank adapter parameterization and quantization, resolving LoRA aggregation conflicts to achieve up to 100x compression with comparable perplexity.

Hang Zou, Chao Zhang, Yuzhi Yang, Yu Tian and 2 more

Published Oct 1, 2026 · 0 citations

Rethinking Token Reweighting for SFT: Suppress, Reverse, and Extrapolate Learned Features

Existing token-reweighting methods cannot reverse harmful SFT features; SCALE uses frozen SFT deltas with entropy-guided gates to suppress, reverse, or extrapolate them, improving math and code results.

Cunchun Li, Haonan He, Yifan Gao, Minglei Li and 3 more

Published Sep 27, 2026 · 0 citations · ▲ 11 on Hugging Face

ScopeIF: Improving Scope-Aware Precise Instruction-Following in Large Language Models via Graded Reward Modeling

ScopeIF improves LLM instruction-following via graded reward modeling and scope-aware constraints, enabling small models to match frontier performance.

Bosi Wen, Yilin Niu, Xiaoying Ning, Ying Zhang and 2 more

Published Sep 26, 2026 · 0 citations

From Personal to Collective: On the Role of Local and Global Knowledge in LLM Personalization

LoGo augments individual user signals with evolving global and community-level behavioral patterns via adaptive mediation, improving LLM personalization and reducing overfitting.

Zehong Wang, Junlin Wu, Zhaoxuan Tan, Bolian Li and 3 more

Published 2026 · 0 citations

Instant Personalized Large Language Model Adaptation via Hypernetwork

A hypernetwork enables instant personalized large language model adaptation by generating user-specific parameters directly from user data.

Zhaoxuan Tan, Zixuan Zhang, Haoyang Wen, Zheng Li and 7 more

Published 2026 · 1 citation

Instruction Tuning for Large Language Models: A Survey

Instruction tuning surveys supervised fine-tuning of LLMs on instruction-output pairs to align next-word prediction with human intent, covering datasets, training, applications, and limitations.

Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang and 7 more

Published Nov 17, 2025 · 79 citations

Instruction Tuning for Story Understanding and Generation with Weak Supervision

Weak to Strong Instruction Tuning improves story understanding and generation by training models on instructions of varying clarity, outperforming state-of-the-art baselines.

Yangshu Yuan, Heng Chen, Christian Ng

Published Jan 26, 2025 · 0 citations

Stronger Models are NOT Stronger Teachers for Instruction Tuning

Stronger models are not stronger teachers for instruction tuning due to teacher-student incompatibility; a compatibility-adjusted reward metric predicts effective generators.

Zhangchen Xu, Fengqing Jiang, Luyao Niu, Lin, Bill Yuchen and 1 more

Published Nov 11, 2024 · 0 citations · ▲ 39 on Hugging Face

Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence

Model Swarms uses swarm intelligence to collaboratively adapt LLM experts in weight space, improving over baselines by up to 21% with minimal data and no tuning.

Shangbin Feng, Zifeng Wang, Yike Wang, Sayna Ebrahimi and 8 more

Published Oct 15, 2024 · 2 citations

QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models

QA-LoRA proposes quantization-aware low-rank adaptation that quantizes LLM weights during fine-tuning and merges adapters into quantized models without accuracy loss.

Yuhui Xu, Lingxi Xie, Xiaotao Gu, Xin Chen and 5 more

Published Sep 26, 2023 · 21 citations · ▲ 46 on Hugging Face · Code ★ 148

LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition

LoraHub dynamically combines existing LoRA modules without extra parameters or gradients to generalize to unseen tasks with few examples, trading some accuracy for much lower inference token costs versus in-context learning.

Chengsong Huang, Qian Liu, Lin, Bill Yuchen, Tianyu Pang and 2 more

Published Jul 25, 2023 · 7 citations · ▲ 34 on Hugging Face · Code ★ 668

SDS-LoRA: Overcoming Anisotropic Gradient Scaling in Low-Rank Adaptation

JungHun Oh, Sungyong Baik, Kyoung Mu Lee

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

LEAN: Library-Based Adaptation for Asynchronous, Federated Fine-Tuning

Erdong Hu, Yuxin Tang, Zhimin Ding, Christopher Jermaine

Paris Poster Session 3, Thu, Dec 10, 12:30 PM–2:30 PM, Paris Poster Hall · Published 2026

ToPA: Block-wise Toeplitz Adaptation for Expressive and Efficient Fine-Tuning

Sicong Li, Qianqian Xu, Zhiyong Yang, Zitai Wang and 3 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Adaptive Fine-Tuning Scheduler for Multi-Tenant Edge LLM via Convergence-Aware Bandits

Yandi Li, Jianxiong Guo, Yupeng Li, Zhiqing Tang and 2 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

VoluCore: Spanning Teacher Representations with Volumetric Coresets for Data-Efficient LLM Distillation

Wang Xi, Yue Wang

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Mask-Conditioned Gradient Masking for Fine-Tuning Mixture-of-Experts Diffusion Language Models

Yiru Tang, Kun Zhou, Xin Zhao, Jing Sha and 2 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Co-evolution: A "One-to-many" LLM Fine-Tuning Paradigm

Dapeng Jiang, Haichuan Tan, Wenxuan Song, Dianqiao Lei and 2 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

LoRAtorio: An intrinsic approach to LoRA Skill Composition

Niki Foteinopoulou, Ignas Budvytis, Stephan Liwicki

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

ENGINE: Endogenous Variational MultiScale Optimization for Zeroth-Order LLM Fine-Tuning

Zhuoli Ouyang, Changxi Chi, Siyuan Li, Tailin Wu

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

FedLoVA: Value-Only Aggregation for Federated LoRA Fine-Tuning of Large Language Models

Ensieh Khazaei, Baturalp Buyukates, Dimitrios Hatzinakos

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Efficient Collaborative LLM Fine-Tuning over Heterogeneous Mobile Devices via Many Backbones to One Side-Network Tuning

Xingke Yang, Liang Li, Sicong Li, Liwei Guan and 5 more

Atlanta Poster Session 5, Fri, Dec 11, 10:00 AM–1:00 PM, Hall C1 · Published 2026

Rethinking LoRA Initialization for Robust Asymmetric Learning Rates

Disen Liao, Yaoliang Yu

Atlanta Poster Session 2, Wed, Dec 9, 4:30 PM–7:30 PM, Hall C1 · Published 2026

Irreducible Supervision Enables Compositional Generalization in Post-Training

Ellen Ma, Nikhil Anand

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

SAFE-DRIFT: Data Selection for Supervised Fine-tuning with Controllable Off-Target Drifts

Yeo Jin Jung, Yating Liu, Lalchand Pandia, Claire Donnat

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

BQ-LoRA: Binary-Quantized Low-Rank Adapters as Implicit Regularizers for Parameter-Efficient Fine-Tuning

Juyoung Park, Heejun Lee

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Beyond Structural Agnosticism: Stable-Rank-Guided LoRA for Structure-Aware Fine-Tuning

Yuanyang Cao, Xichun Liu, Fuwei Zhang, Meiqin Liu and 1 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Beyond Decoupled PEFT: Geometry-Aware Low-Rank Adaptation via Riemannian Reparameterization

Yuanyang Cao, Xichun Liu, Haitao Jiang, Jianji Wang

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Fine-tuning Does Not Reach All: Uneven Safety and Knowledge Dynamics in Language Models

Ziwei Wang, Xinwei Guo, Jiaxin Zhang, Guanhua Chen and 6 more

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

No Model Required: Text Entropy Rate Filtering Prevents Iterative Fine-Tuning Collapse

Lewis Mitchell

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

SAFTAC: Simulation-Augmented Fine-Tuning of Open-Source LLMs for Analog Circuit Design

Junsheng Huang, Yifan Sun, Zhuoer Zhang, Ning Wei and 6 more

Atlanta Poster Session 2, Wed, Dec 9, 4:30 PM–7:30 PM, Hall C1 · Published 2026

HALO: Heterogeneous-Aware LoRA Optimization via Rank Allocation and Client-Aware Projection

XiaoHua Feng, Yuyuan Li, Jiayuan Fang, Fengyuan Yu and 5 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Aligned LoRA Updates via Intrinsic Geometry for Federated Low-Rank Adaptation

Yihao Yang, Jian Liang, Wenke Huang, He Li and 2 more

Paris Poster Session 1, Wed, Dec 9, 12:30 PM–2:30 PM, Paris Poster Hall · Published 2026

DISTMOE: Rehearsal-free Routing in Mixture-of-Experts for Distributed Instruction Tuning

Mainak Singha, Niccolò Biondi, Elisa Ricci, Subhankar Roy

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Geometry-Aware Zeroth-Order Optimization for Fine-Tuning Quantized LLMs

Shaocong Ma, Weidong Cai, Heng Huang

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

FLoRA-Chef: Making A Good LoRA Recipe in Federated Generalization

Wenwen He, Wenke Huang, Yiyan Qi, He Li and 3 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Modulating Merging Strengths via Joint Loss Estimation for LoRA-based Continual Learning

Kun Gu, De Cheng, Zhipeng Xu, Lingfeng He and 2 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026