arXiv 2402.04624
MEMORYLLM: Towards Self-Updatable Large Language Models
By Yu Wang, Yifan Gao, et al.
Published 2024-02-07
Wiki summary
Explore the paper's summary, context, and related research on Papiers.
Existing Large Language Models (LLMs) usually remain static after deployment, which might make it hard to inject new knowledge into the model. We aim to build models containing a considerable portion of self-updatable parameters, enabling the model to integrate new knowledge effectively and efficiently. To this end, we introduce MEMORYLLM, a model that comprises a transformer and a fixed-size memory pool within the…