arXiv 2305.13048
RWKV: Reinventing RNNs for the Transformer Era
By Bo Peng, Eric Alcaide, et al.
Published 2023-05-22
Wiki summary
Explore the paper's summary, context, and related research on Papiers.
Transformers have revolutionized almost all natural language processing (NLP) tasks but suffer from memory and computational complexity that scales quadratically with sequence length. In contrast, recurrent neural networks (RNNs) exhibit linear scaling in memory and computational requirements but struggle to match the same performance as Transformers due to limitations in parallelization and scalability. We propose…