arXiv 2509.00421
Memory Limitations of Prompt Tuning in Transformers
By Maxime Meyer, Mario Michelessa, et al.
Published 2025-08-30
Discussion
Read the public discussion and references gathered around this paper.
Despite the empirical success of prompt tuning in adapting pretrained language models to new tasks, theoretical analyses of its capabilities remain limited. Existing theoretical work primarily addresses universal approximation properties, demonstrating results comparable to standard weight tuning. In this paper, we explore a different aspect of the theory of transformers: the memorization capability of prompt tuning…