arXiv 2512.24880
mHC: Manifold-Constrained Hyper-Connections
By Zhenda Xie, Yixuan Wei, et al.
Published 2025-12-31
Wiki summary
Explore the paper's summary, context, and related research on Papiers.
Recently, studies exemplified by Hyper-Connections (HC) have extended the ubiquitous residual connection paradigm established over the past decade by expanding the residual stream width and diversifying connectivity patterns. While yielding substantial performance gains, this diversification fundamentally compromises the identity mapping property intrinsic to the residual connection, which causes severe training ins…