arXiv 2506.19830
Scaling Speculative Decoding with Lookahead Reasoning
By Yichao Fu, Rui Ge, et al.
Published 2025-06-24
Wiki summary
Explore the paper's summary, context, and related research on Papiers.
Reasoning models excel by generating long chain-of-thoughts, but decoding the resulting thousands of tokens is slow. Token-level speculative decoding (SD) helps, but its benefit is capped, because the chance that an entire -token guess is correct falls exponentially as grows. This means allocating more compute for longer token drafts faces an algorithmic ceiling -- making the speedup modest and hardware-agnostic. We…