arXiv 2608.17981
Recirculation
By Michael C. Mozer, Shoaib Ahmed Siddiqui, et al.
Published 2026-08-18
Discussion
Read the public discussion and references gathered around this paper.
We describe an inference-time architectural enhancement for off-the-shelf foundation models that markedly reduces perplexity and boosts accuracy across generation and reasoning tasks. Our approach incurs essentially no additional latency during generation, though it requires serial processing in the prefill phase. Motivated by the fundamental limitation that state updates in feedforward transformers are bounded by m…