Ai2 releases Olmo-core 3 to make developing large mixture-of-experts LLMs more efficient

Ai2 releases Olmo-core 3 to make developing large mixture-of-experts LLMs more efficient

Seattle-based artificial intelligence research firm Allen Institute for AI announced a development framework for large language models Thursday that significantly improves how mixture-of-experts large language models are trained.

The new framework, Olmo-core 3, allows MoE training to reach the trillion-parameter scale while keeping costs low by preserving computational efficiency.

Mixture-of-experts models operate differently from dense AI models: they split computation across specialist portions of the model each time…