arXiv 2501.00070
ICLR: In-Context Learning of Representations
By Core Francisco Park, Andrew Lee, et al.
Published 2024-12-29
Citation lineage
Review the prior work and downstream research connected to this paper.
Recent work has demonstrated that semantics specified by pretraining data influence how representations of different concepts are organized in a large language model (LLM). However, given the open-ended nature of LLMs, e.g., their ability to in-context learn, we can ask whether models alter these pretraining semantics to adopt alternative, context-specified ones. Specifically, if we provide in-context exemplars wher…