Llama 4 Scout
By Meta
Llama 4 Scout is an open-weight multimodal large language model from Meta built on a mixture-of-experts architecture, designed to offer a very long context window and efficient inference by activating only a subset of its total parameters…
Definition
Llama 4 Scout is an open-weight multimodal large language model from Meta built on a mixture-of-experts architecture, designed to offer a very long context window and efficient inference by activating only a subset of its total parameters for each request. Its mixture-of-experts design activates only a subset of the model's total parameters for each token, which keeps per-token compute low while still supporting a very large context window suited to reasoning over lengthy documents or codebases.
Overview
Llama 4 Scout is part of Meta's Llama 4 generation, which introduced a mixture-of-experts, or MoE, architecture to the Llama family for the first time. In an MoE model, the network is divided into multiple specialized expert sub-networks, and a routing mechanism activates only a small subset of those experts for any given input, which allows the model to have a large total parameter count while keeping the computation required per token comparatively low. A trained router network decides, for each token, which handful of experts among the model's total pool should process it, so the effective computation per token stays close to that of a much smaller dense model even though the total parameter count stored on disk is far larger. Scout is positioned as the more efficient, longer-context member of the Llama 4 lineup, supporting a very large context window intended for tasks that require reasoning over extensive documents, codebases, or multi-document collections in a single pass, distinguishing it from the multimodal Llama 3.2 models that had more modest context limits. Scout's very long context window is paired with the mixture-of-experts efficiency gain specifically so that processing a long document does not multiply compute cost the way it would in an equivalently sized dense model, making it comparatively practical to run long-context workloads that would otherwise require a much larger active-parameter budget. As a multimodal model, Llama 4 Scout accepts both text and image inputs, continuing the direction started with Llama 3.2's vision-capable variants but built on the newer MoE backbone, letting it combine visual understanding with the efficiency benefits of sparse expert activation. Within the Llama 4 generation, Scout's emphasis on context length and lower active-parameter cost per token contrasts with Maverick's larger active-parameter budget aimed at raw capability, so the two effectively split the trade-off space between efficient long-context handling and maximum quality per request. Llama 4 Scout is released with open weights under Meta's licensing terms, allowing self-hosted deployment and fine-tuning, though running an MoE model efficiently typically requires infrastructure capable of handling the routing and memory layout of multiple experts, which can differ from serving a standard dense transformer model. Teams processing large codebases, lengthy legal or technical documents, or multi-document research collections have used Scout to reason across material that would need to be chunked and stitched together with a shorter-context model, while its multimodal support lets the same pipeline also handle embedded images or diagrams in that material. Within the Llama 4 family, Scout is generally positioned as the more lightweight and efficiency-oriented option compared to the larger Llama 4 Maverick model, trading some raw capability for lower inference cost and a stronger emphasis on very long context handling. Serving a mixture-of-experts model well requires infrastructure that can route requests to the correct experts and manage memory across them, which is a meaningfully different operational burden than serving a dense transformer, so teams without existing mixture-of-experts serving experience should budget time for that adjustment before deploying Scout in production.
Key Features
- Mixture-of-experts architecture with sparse per-token activation
- Very long context window for large document and codebase tasks
- Multimodal support for combined text and image inputs
- Open weights available for self-hosted deployment and fine-tuning
- Lower per-token inference cost relative to a dense model of similar total size
- Positioned as the more efficiency-focused sibling to Llama 4 Maverick