OLMo
By Allen Institute for AI (Ai2)
OLMo (Open Language Model) is a fully open large language model project from the Allen Institute for AI (Ai2), notable for releasing not just model weights but also training data, code, and training logs, in contrast to most "open-weight"…
Definition
OLMo (Open Language Model) is a fully open large language model project from the Allen Institute for AI (Ai2), notable for releasing not just model weights but also training data, code, and training logs, in contrast to most "open-weight" models that withhold their training data.
Overview
OLMo was launched by Ai2 in February 2024 specifically to address a gap in the open-model ecosystem: most models labeled "open source," including Llama and Mistral, release only model weights and sometimes inference code, without disclosing the training data or full training recipe needed to reproduce or deeply audit the model. OLMo instead released everything — model weights, the full pretraining dataset (Dolma, a roughly 3-trillion-token corpus that Ai2 also released separately), training and evaluation code, and intermediate checkpoints — explicitly to support scientific research into how LLMs work and how choices in data and training affect behavior. The initial OLMo release included 1B and 7B parameter models. Ai2 followed with OLMo 2 in late 2024, improving training stability and benchmark performance to be competitive with similarly sized open-weight models like Llama 2/3 and Mistral's smaller releases, while maintaining the same full-transparency release philosophy. Ai2 also released OLMoE, a mixture-of-experts variant, and multimodal extensions, continuing to prioritize reproducibility alongside capability. OLMo is positioned less as a commercial product and more as public research infrastructure — it is widely cited in interpretability, data-attribution, and training-dynamics research precisely because outside researchers can trace model behavior back to specific training data and checkpoints, something impossible with closed-data models. It sits alongside other fully open efforts like EleutherAI's Pythia and LLM360's models as part of a small but important segment of the field dedicated to reproducible, auditable LLM research, distinct from the much larger set of open-weight-only releases from companies like Meta, Mistral AI, and Alibaba.
Key Concepts
- Releases full training data (the Dolma corpus), not just model weights
- Publishes training code, evaluation code, and intermediate checkpoints
- Designed explicitly to support reproducible LLM research
- OLMo 2 improved training stability and benchmark competitiveness
- OLMoE variant explores a mixture-of-experts architecture openly
- Released under fully open licenses with no data opacity
- Backed by the Allen Institute for AI, a nonprofit research institute