arXiv 2601.19897
Self-Distillation Enables Continual Learning
By Idan Shenfeld, Mehul Damani, et al.
Published 2026-01-27
Citation lineage
Review the prior work and downstream research connected to this paper.
Continual learning, enabling models to acquire new skills and knowledge without degrading existing capabilities, remains a fundamental challenge for foundation models. While on-policy reinforcement learning can reduce forgetting, it requires explicit reward functions that are often unavailable. Learning from expert demonstrations, the primary alternative, is dominated by supervised fine-tuning (SFT), which is inhere…