Hugging Face AI Open-Source Blog: Introducing Olmo-core 3: Open, scalable training infrastructure for large MoEs
Ai2 released Olmo-core 3, an open-source training framework for large-scale Mixture of Experts (MoE) models. Its design goal is to scale MoE training to the trillions-of-parameters level and serves as one of the core training systems for the next-generation Olmo models.
Return to original text
A short introduction is retained here; the full text from the source website is not reposted. For specific wording and subsequent updates, please refer to the original article.
Open Hugging Face blog original