arXiv 2512.05117
The Universal Weight Subspace Hypothesis
By Prakhar Kaushik, Shravan Chaudhari, et al.
Published 2025-12-04
Discussion
Read the public discussion and references gathered around this paper.
We show that deep neural networks trained across diverse tasks exhibit remarkably similar low-dimensional parametric subspaces. We provide the first large-scale empirical evidence that demonstrates that neural networks systematically converge to shared spectral subspaces regardless of initialization, task, or domain. Through mode-wise spectral analysis of over 1100 models - including 500 Mistral-7B LoRAs, 500 Vision…