SynthCommonCorpus: the largest open synthetic pretraining corpus built for agentic models

Talk overview

This talk introduces SynthCommonCorpus, an open synthetic pretraining corpus designed for agentic models and planned for release in autumn 2026. It incorporates tool calls, multi-step task decomposition, and adversarial examples into pretraining data to study how models learn to act. The session explains how synthetic generation and traceable, licensed seed data support reproducibility and the measurement of memorization and data contamination.