arXiv 2510.25000
What Really Matters in Matrix-Whitening Optimizers?
By Kevin Frans, Pieter Abbeel, et al.
Published 2025-10-28
Citation lineage
Review the prior work and downstream research connected to this paper.
A range of recent optimizers have emerged that approximate the same "matrix-whitening" transformation in various ways. In this work, we systematically deconstruct such optimizers, aiming to disentangle the key components that explain performance. Across tuned hyperparameters across the board, all flavors of matrix-whitening methods reliably outperform elementwise counterparts, such as Adam. Matrix-whitening is often…