mHC: Manifold-Constrained Hyper-Connections
[Submitted on 31 Dec 2025 (v1), last revised 5 Jan 2026 (this version, v2)] / arXiv.org
Recently, studies exemplified by Hyper-Connections (HC) have extended the ubiquitous residual connection paradigm established over the past decade by expanding the residual stream width and diversifying connectivity patterns. While yielding substantial performance gains, this diversification fundamentally compromises the identity mapping property intrinsic to the residual connection, which causes severe training instability and restricted scalability, and additionally incurs notable memory acces | -- |