Open Source Models & Infra
The Smol Training Handbook: One Tradeoff, Any Scale
Date, time, and room will be added once confirmed.
Talk overview
Training state-of-the-art LLMs involves balancing memory, compute, and communication across different scales. The talk explores parallelism strategies like data, context, and pipeline parallelism, emphasizing the importance of infrastructure understanding and practical open-source tools. It concludes with guidance on selecting the right strategy and profiling techniques for production environments.
