arXiv 2502.21074
CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation
By Zhenyi Shen, Hanqi Yan, et al.
Published 2025-02-28
Citation lineage
Review the prior work and downstream research connected to this paper.
Chain-of-Thought (CoT) reasoning enhances Large Language Models (LLMs) by encouraging step-by-step reasoning in natural language. However, leveraging a latent continuous space for reasoning may offer benefits in terms of both efficiency and robustness. Prior implicit CoT methods attempt to bypass language completely by reasoning in continuous space but have consistently underperformed compared to the standard explic…