arXiv 2402.04624
MEMORYLLM: Towards Self-Updatable Large Language Models
By Yu Wang, Yifan Gao, et al.
Published 2024-02-07
Mindmap
Browse the paper's core ideas, clusters, and relationships in a structured outline.
Existing Large Language Models (LLMs) usually remain static after deployment, which might make it hard to inject new knowledge into the model. We aim to build models containing a considerable portion of self-updatable parameters, enabling the model to integrate new knowledge effectively and efficiently. To this end, we introduce MEMORYLLM, a model that comprises a transformer and a fixed-size memory pool within the…