arXiv 2501.00070
ICLR: In-Context Learning of Representations
By Core Francisco Park, Andrew Lee, et al.
Published 2024-12-29
Mindmap
Browse the paper's core ideas, clusters, and relationships in a structured outline.
Recent work has demonstrated that semantics specified by pretraining data influence how representations of different concepts are organized in a large language model (LLM). However, given the open-ended nature of LLMs, e.g., their ability to in-context learn, we can ask whether models alter these pretraining semantics to adopt alternative, context-specified ones. Specifically, if we provide in-context exemplars wher…