arXiv 2405.06394
Memory Mosaics
By Jianyu Zhang, Niklas Nolte, et al.
Published 2024-05-10
Mindmap
Browse the paper's core ideas, clusters, and relationships in a structured outline.
Memory Mosaics are networks of associative memories working in concert to achieve a prediction task of interest. Like transformers, memory mosaics possess compositional capabilities and in-context learning capabilities. Unlike transformers, memory mosaics achieve these capabilities in comparatively transparent way ("predictive disentanglement"). We illustrate these capabilities on a toy example and also show that me…