arXiv 2601.19897
Self-Distillation Enables Continual Learning
By Idan Shenfeld, Mehul Damani, et al.
Published 2026-01-27
Mindmap
Browse the paper's core ideas, clusters, and relationships in a structured outline.
Continual learning, enabling models to acquire new skills and knowledge without degrading existing capabilities, remains a fundamental challenge for foundation models. While on-policy reinforcement learning can reduce forgetting, it requires explicit reward functions that are often unavailable. Learning from expert demonstrations, the primary alternative, is dominated by supervised fine-tuning (SFT), which is inhere…