QK-Wanda: Coupling Queries and Keys for Unstructured Pruning

QK-Wanda couples query and key pruning scores via cross-projection deletion costs, reducing QK reconstruction error by 60% at 50% sparsity and improving downstream perplexity on some large models.

Ivan Ilin, Peter Richtárik

Published Oct 1, 2026 · 0 citations

Efficient Fine-Tuning for Structured Sparsity Under Group Repartitioning

Diyang Li

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

ASAP: Attention Sink Anchored Pruning

Jaehyuk Lee, Hanyoung Kim, Yanggee Kim, Donghun Lee

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

A Free Lunch in LLM Compression: Revisiting Retraining after Pruning

Moritz Wagner, Christophe Roux, Max Zimmer, Sebastian Pokutta

Paris Poster Session 6, Fri, Dec 11, 2:30 PM–4:30 PM, Paris Poster Hall · Published 2026

Compressible Representations: Functional Spines in Deep Neural Networks

Niranjan Rajesh, Meenakshi Khosla

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

SKIM: Pruning Large Language Model Agents via Selective Knowledge Informed Masking

Moonseok Choi, Giung Nam, Jongwon Jeong, Minki Kang and 1 more

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

TOM-Pruning: Target-aware Output Manifold for LLM Pruning

Junchen Hao, Weikang Meng, Yingjian Li, Zheng Zhang

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

AC/DC on a Budget -- Alternating Sparse Phases

Rahul Nittala, Advait Gadhikar, Tom Jacobs, Rebekka Burkholz

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

ResKV: Residual-based Channel-wise Unstructured Pruning for KV Cache Compression

Yue Chen, Jinze Li, Dajiang Liu

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Scattered by Design: Why Per-Output Pruning Resists Structured Compression

Victor Omolaoye, Gerard de Melo

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Constrained Factorization with Diagonal Scaling: Rank-Revealing Training and Pruning

Yikun Hou, Emrullah Akbas, Suvrit Sra, Alp Yurtsever

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Block-OBS-GS: Exact Per-Block Joint Brain Surgery with Gauss–Seidel Refinement for LLM Pruning

Yuwen Huang, Xiang Pan

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Generation-Drift-Guided Block Pruning for Large Language Models

ZHIQI HUANG, Heedong Kim, ZHANG LINTONG, Seong-Whan Lee

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Cannistraci-Hebb Channel-wise Dynamic Sparse Training of Convolutional Neural Networks with Contextual Modulation

Wenjing Wu, Hanming Li, Xizheng Deng, Jialin Zhao and 3 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Correspondence Pruning by Iterative Structural Rectification

Guangwei Zhang, Gang Wang

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Sparsity for Free: A Budget-Induced Equilibrium in Joint Topology–Parameter Search

Marcel Mordarski, Daniel Budina, Benjamin Gras, Abdulrahman Shehata and 1 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Sparsely Wired Mortal LLM Inference

Mincheol Park, Sunwoo Lee, Sukjin Lee, Wooram Yang and 3 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Fewer Tokens, Fewer Layers: Efficient Vision Token Pruning and On-Policy Distillation to Accelerate VLMs

Shuai Wang, Shitong Shao, Qi Xuan, Zhaowei Zhu and 1 more

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

TrunkFish: Making Model Width Incrementally Refinable

Owen M Dugan, Liam Dugan, Aaryan Singhal, Christopher De Sa and 1 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

SVDP: Training-Free Contextual Sparsity Predictors for Fast LLM Inference

Georgii Serbin, Kirill Koshkin, Zhongao Sun, Anastasiya Bistrigova and 1 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

LIPAR: Latent Inter-Frame Pruning with Attention Recovery

Dennis Y Menn, Yuedong Yang, Bokun Wang, Xiwen Wei and 5 more

Atlanta Poster Session 1, Wed, Dec 9, 10:00 AM–1:00 PM, Hall C1 · Published 2026

Unifying Sparsity and Discreteness: One-Shot Pruning for Quantized LLMs via Discrete Optimization

Haozhen Zhang, Hanyuan Zheng, Teng Hou, Zhaogeng Liu and 2 more

Paris Poster Session 4, Thu, Dec 10, 5:30 PM–7:30 PM, Paris Poster Hall · Published 2026

Position: Lottery Tickets Do Not Explain Overparameterization. How About Escape Dimensions?

Flavio Martinelli, Johanni Brea, Wulfram Gerstner

Paris Poster Session 1, Wed, Dec 9, 12:30 PM–2:30 PM, Paris Poster Hall · Published 2026

SliMOO: Interpretable Multi-Objective Evolutionary Search for LLM Depth Pruning

Guanchen Li, Yixing Xu, Xuanwu Yin, Dong Li and 1 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Beyond Masked Sparsity: SNACK Enables Truly Sparse Neural Networks on GPU

Jafar Badour, Elena Mocanu, Maurice Keulen

Paris Poster Session 4, Thu, Dec 10, 5:30 PM–7:30 PM, Paris Poster Hall · Published 2026

On the Efficiency of Structured Pruning in Small Language Model Pretraining

Yixiao Li, Xianzhi Du, AJAY JAISWAL, Tao Lei and 3 more

Atlanta Poster Session 3, Thu, Dec 10, 10:00 AM–1:00 PM, Hall C1 · Published 2026

FiedlerPrune: Connectivity-Preserving Cross-Layer Pruning for Large Language Models

Zijun Sun, Yanning Shen

Atlanta Poster Session 2, Wed, Dec 9, 4:30 PM–7:30 PM, Hall C1 · Published 2026

When and What to Prune? Stage-Aware Visual Token Pruning for Efficient VLA

TIANJUN SHI, Haotian Xiong, Ziyu Gong, Qi Lu and 1 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

How Much of a Model Do We Need? Redundancy and Slimmability in Remote Sensing Foundation Models

Leonard Hackel, Begum Demir, Tom Burgert

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

EventPrune: Cascaded Event-Assisted Token Pruning for Efficient First-Person Dynamic Spatial Reasoning

Event Cascade Pruning uses event-camera motion cues to prune video tokens in first-person spatial reasoning, improving accuracy by 1.31 points with 80% fewer tokens and 1.89x speedup.

Pengtao Ma, Ziliang Zhou, Ciyu Ruan, Haoyang Wang and 6 more

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

Dense Structural Compression of Transformers via Gauge-Correct Channel Removal

GaugeLasso uses gauge-correct channel penalties to structurally compress transformers during training, reducing compute up to 255x with preserved accuracy and outperforming hand-designed baselines.

Jed A Duersch, Naïm Es-sebbani, Nathanaël Haas, Zied Bouraoui

Paris Poster Session 3, Thu, Dec 10, 12:30 PM–2:30 PM, Paris Poster Hall · Published 2026

PARE: Pruning and Adaptive Routing for Efficient Video Generation

PARE combines structure-aware width pruning and timestep-conditioned adaptive depth routing to cut video diffusion compute while preserving generation quality.

Yutong Wang, Yunke Wang, Tianfan Xue, Yu Qiao and 3 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Importance-Aware OBS Pruning for Diffusion Models

Importance-aware pruning for diffusion models uses spatial importance maps to retain parameters critical to semantically salient regions, preserving subject fidelity and structural correctness at high compression ratios.

Ba-Thinh Lam, Srijan Das, Hieu Le

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Spectral-Aligned Pruning for Universal Error-Correcting Code Transformers

Spectral-Aligned Pruning uses code graph eigenvalues to retrieve reusable structured pruning masks for universal error-correcting transformers, recovering accuracy via LoRA adapters to cut computation and memory.

Sanghyeon Cho, Taewoo Park, Seong-Joon Park, Dae-Young Yun and 3 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

The Sparsity Whisperer

Difference-informed pruning preserves output differences via difference-aware weight scoring, improving LLM sparsity over activation and reconstruction baselines at minimal cost.

Linghao Kong, Inimai Subramanian, Micah Adler, Dan Alistarh and 2 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

CORP: Closed-Form One-shot Representation-Preserving Structured Pruning for Transformers

CORP uses closed-form ridge regression to recover representations and prune transformer structures without retraining, retaining 83.27% ImageNet accuracy at 50% sparsity.

Boxiang Zhang, Baijian Yang

Atlanta Poster Session 3, Thu, Dec 10, 10:00 AM–1:00 PM, Hall C1 · Published 2026

PATCH: Learnable Tile-Level Hybrid Sparsity for LLMs

PATCH learns tile-level hybrid sparsity mixing dense and 2:4 tiles for LLMs, enabling tunable sparsity ratios that improve accuracy and deliver 1.18x-1.38x speedups over dense models.

Mohammad Mozaffari, Younes Hourri, Maryam Mehri Dehnavi

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026 · ▲ 1 on Hugging Face

POP: Online Structural Pruning Enables Efficient Inference of Large Foundation Models

POP enables context-conditioned online structural pruning of foundation models via coarse-to-fine partitioned masking without offline calibration or retraining, improving accuracy with lower latency.

Yi Chen, Wonjin Shin, Shuhong Liu, Tho Mai and 5 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

Provable Pruning for Efficient 3D Gaussian Splatting via Coresets

This work proves 3D Gaussian Splatting permits small weighted coresets with resolution-dependent multiplicative guarantees via sensitivity sampling, enabling aggressive pruning without costly finetuning.

Waseem Mousa, Alaa Maalouf

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models

FlexMoE ranks and prunes Mixture-of-Experts channels via discrete actions to generate nested subnetworks across budgets, preserving ~99.8% performance at 50% pruning without fine-tuning.

Fan Mo, Han Yuxuan, Geng Zhang, Wangbo Zhao and 1 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026 · ▲ 1 on Hugging Face

Compact SO(3) Equivariant Atomistic Foundation Models via Structural Pruning

Structural pruning of SO(3) equivariant atomistic models reduces inference cost while outperforming small from-scratch models on accuracy and compute.

Chen Wang, Siyu Hu, Guangming Tan, Weile Jia

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

ROCKET: Rapid Optimization via Calibration-guided Knapsack-Enhanced Truncation for Efficient Model Compression

ROCKET is a training-free compression method using knapsack-based layer allocation and calibration-guided sparse factorization to cut model sizes 20-50% with minimal accuracy loss.

Ammar Ali, Baher Mohammad, Denis Makhov, Dmitriy Shopkhoev and 2 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026 · ▲ 18 on Hugging Face · Code ★ 25

GeoPair: Geometry-Preserving Cross-Layer Factorization for Training-Free Transformer Compression

GeoPair sequentially optimizes cross-layer weight pairings and shared-dictionary factorizations to preserve layer-specific activation geometry for training-free transformer compression, achieving state-of-the-art results.

Baher Mohammad, Ammar Ali, Stamatios Lefkimmiatis

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026 · ▲ 17 on Hugging Face