Open Source Models & Infra
Rethinking Residual Connections: Toward Depth-Effective Language Model Scaling
Date, time, and room will be added once confirmed.
Talk overview
Researchers explore whether recent residual-connection designs can enable effective depth scaling in language models. They evaluate various methods across different model depths and parameter scales, finding that only some designs sustain meaningful contributions from deeper layers. These results highlight residual architecture as a key factor in model scaling, alongside width, data, and compute.
