arXiv 2510.15511
Language Models are Injective and Hence Invertible
By Giorgos Nikolaou, Tommaso Mencattini, et al.
Published 2025-10-17
Wiki summary
Explore the paper's summary, context, and related research on Papiers.
Transformer components such as non-linear activations and normalization are inherently non-injective, suggesting that different inputs could map to the same output and prevent exact recovery of the input from a model's representations. In this paper, we challenge this view. First, we prove mathematically that transformer language models mapping discrete input sequences to their corresponding sequence of continuous r…