mHC: Manifold-Constrained Hyper-Connections
Zhenda Xie, Yixuan Wei, Huanqi Cao, et al.
A framework that addresses training instability and memory overhead in Hyper-Connections architectures by projecting residual connection space onto specific manifolds. Demonstrates improved scalability for large-scale model training.