arXiv 2112.00114
Show Your Work: Scratchpads for Intermediate Computation with Language Models
By Maxwell Nye, Anders Johan Andreassen, et al.
Published 2021-11-30
Citation lineage
Review the prior work and downstream research connected to this paper.
Large pre-trained language models perform remarkably well on tasks that can be done "in one pass", such as generating realistic text or synthesizing computer programs. However, they struggle with tasks that require unbounded multi-step computation, such as adding integers or executing programs. Surprisingly, we find that these same models are able to perform complex multi-step computations -- even in the few-shot re…