SmolLM
By Hugging Face
7 billion parameters, designed to deliver strong performance for on-device and resource-constrained deployment while training on a fully public, high-quality dataset.
Definition
SmolLM is a family of small, fully open-source language models from Hugging Face, ranging from 135 million to 1.7 billion parameters, designed to deliver strong performance for on-device and resource-constrained deployment while training on a fully public, high-quality dataset.
Overview
SmolLM was released by Hugging Face in mid-2024 as part of its broader push to demonstrate that carefully curated, high-quality training data can let very small language models punch above their weight class. The initial family spanned three sizes — 135M, 360M, and 1.7B parameters — all trained on SmolLM-Corpus, a meticulously filtered and deduplicated dataset combining synthetic textbook-style content (Cosmopedia), deduplicated educational web text (FineWeb-Edu), and curated Python code, rather than relying on raw, unfiltered web scrapes. Hugging Face followed with SmolLM2 in late 2024, which improved training data quality and scale further, and expanded the ecosystem with related projects like SmolVLM (a small vision-language model) and SmolAgents (a lightweight agent framework), all sharing the goal of making capable AI accessible on consumer hardware — laptops, phones, and even browsers via WebGPU — without requiring cloud infrastructure. Every component of SmolLM, including model weights, training code, and the training dataset itself, was released fully openly, distinguishing it from many 'open-weight' models that release only the final weights without the data or training recipe. SmolLM models are commonly used as a reference point for research into data-efficient small-model training, as building blocks for on-device applications where privacy or offline operation matters, and as base models for further fine-tuning in resource-constrained settings. They compete in the same small-model category as Microsoft's Phi series, Google's Gemma, and Mistral's Ministral models, with SmolLM's particular differentiation being its fully open data and training pipeline alongside the model weights.
Key Concepts
- Fully open release: model weights, training code, and training dataset all public
- Model sizes ranging from 135M to 1.7B parameters
- Trained on SmolLM-Corpus: curated synthetic textbooks, deduplicated educational web text, and code
- SmolLM2 improved data quality and scale over the original release
- Designed to run on consumer hardware, including browsers via WebGPU
- Part of a broader open small-model ecosystem including SmolVLM and SmolAgents
- Popular as a research reference for data-efficient small-model training