arXiv 2510.15103
Continual Learning via Sparse Memory Finetuning
By Jessy Lin, Luke Zettlemoyer, et al.
Published 2025-10-16
Citation lineage
Review the prior work and downstream research connected to this paper.
Modern language models are powerful, but typically static after deployment. A major obstacle to building models that continually learn over time is catastrophic forgetting, where updating on new data erases previously acquired capabilities. Motivated by the intuition that mitigating forgetting is challenging because trainable parameters are shared across all tasks, we investigate whether sparse parameter updates can…