arXiv 2510.15511
Language Models are Injective and Hence Invertible
By Giorgos Nikolaou, Tommaso Mencattini, et al.
Published 2025-10-17
Mindmap
Browse the paper's core ideas, clusters, and relationships in a structured outline.
Transformer components such as non-linear activations and normalization are inherently non-injective, suggesting that different inputs could map to the same output and prevent exact recovery of the input from a model's representations. In this paper, we challenge this view. First, we prove mathematically that transformer language models mapping discrete input sequences to their corresponding sequence of continuous r…