Code Llama
Meta's code-specialized Llama variant
Code Llama is a family of large language models from Meta, built by further training Llama 2 on code-heavy data, specialized for code generation, completion, and understanding, and released in multiple sizes and variants including…
Definition
Code Llama is a family of large language models from Meta, built by further training Llama 2 on code-heavy data, specialized for code generation, completion, and understanding, and released in multiple sizes and variants including Python-specific and instruction-tuned versions.
Overview
Code Llama takes Meta's general-purpose Llama 2 foundation model and continues training it on a large corpus dominated by publicly available source code, along with some natural-language data related to code such as discussions and documentation. This continued pretraining approach lets Code Llama inherit Llama 2's general language capability while substantially improving its accuracy and fluency on programming tasks compared to the base model. The family was released in three sizes — 7B, 13B, and 34B parameters (with a later 70B variant added) — and in several specialized flavors: the base Code Llama models for general code completion, Code Llama - Python, further specialized for Python specifically, and Code Llama - Instruct, fine-tuned to follow natural-language instructions for tasks like explaining code or generating code from a description. This structure lets developers choose a variant matched to their exact use case, whether that's raw code completion in an IDE or instruction-driven code generation in a chat interface. Code Llama supports long-context input via a technique called fill-in-the-middle training, which allows the model to complete code given both preceding and following context (not just what comes before), which is particularly useful for IDE-style autocomplete where the cursor sits in the middle of existing code. Released with open weights under Meta's community license, Code Llama has been widely adopted as a base for fine-tuned coding assistants and as a benchmark reference point for subsequent open code models such as StarCoder2 and WizardCoder.
Key Features
- Built by continued pretraining of Llama 2 on code-heavy data
- Released in 7B, 13B, 34B, and later 70B parameter sizes
- Specialized Python and Instruct variants alongside the base models
- Fill-in-the-middle training for cursor-aware code completion
- Long context window suited to multi-file code tasks
- Open-weight release under Meta's community license
- Widely used as a base for fine-tuned coding assistants
- Strong benchmark performance on code-generation tasks like HumanEval
Use Cases
Alternatives
Frequently Asked Questions
From the Blog
How to Sandbox an Agent That Executes Code
Model-generated code must run as untrusted input: in a container with no host mounts, a default-deny network, a non-root user, a read-only filesystem, and hard limits on memory, processes and wall-clock time. This article covers each control and the failure it prevents.
Read More AI & TechnologyHow to Version and Test Prompts Like Application Code
Treat prompts as versioned artefacts: keep them in files under source control, pin the version used by each deployment, review changes as diffs, and gate merges on an evaluation suite. The result is that any output can be traced to the exact prompt that produced it, and any regression can be reverted.
Read More AI & TechnologyHow to review AI-generated code without rubber-stamping it
AI code fails differently from human code: clean style, invented assumptions. Use a review order checking interfaces, invented APIs and edge cases first.
Read More AI & TechnologySecurity and licence risks in AI-generated code, and how to catch them
The risk is ordinary insecure defaults arriving faster than review. Learn the patterns to scan for, how secrets leak through prompts, and which gates to automate.
Read More