ACL 2025LLM reasoning & chain of thought Nanyang Technological Joint NTU-UBC Research Centre of

SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs

SoftCoT uses a fixed assistant and projection module to generate soft reasoning tokens that boost LLM reasoning via parameter-efficient fine-tuning.

Yaogeng Xu, Xu Guo, Zhiwei Zeng, Chunyan Miao

Published 202514 citationsPaper ↗

76%

OverallHighly rated

OverallHighly ratedVote to see the scoreThe exact score shows once you've voted, so every vote is your own call. The first half of each home page shelf shows its scores.

Readers

–

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel10/20reviewers recommend it

lenient 4/5

medium 5/10

strict 1/5

AI panel?Vote to see what the 20 AI reviewers said

Panel consensus

SoftCoT enables continuous reasoning without forgetting via frozen LLMs, soft tokens, and parameter-efficient projection, though unverified latency costs and narrow benchmark gains leave its larger-scale durability uncertain.

Abstract

Chain-of-Thought (CoT) reasoning enables Large Language Models (LLMs) to solve complex reasoning tasks by generating intermediate reasoning steps.However, most existing approaches focus on hard token decoding, which constrains reasoning within the discrete vocabulary space and may not always be optimal.While recent efforts explore continuousspace reasoning, they often require full-model fine-tuning and suffer from catastrophic forgetting, limiting their applicability to state-of-theart LLMs that already perform well in zeroshot settings with a proper instruction.To address this challenge, we propose a novel approach for continuous-space reasoning that does not require modifying the LLM.Specifically, we employ a lightweight fixed assistant model to speculatively generate instancespecific soft thought tokens as the initial chain of thoughts, which are then mapped into the LLM's representation space via a trainable projection module.Experimental results on five reasoning benchmarks demonstrate that our method enhances LLM reasoning performance through supervised, parameter-efficient fine-tuning.Source code is available at https: //github.com/xuyige/SoftCoT.