CodeLlama
By Meta
Code Llama is a family of large language models specialized for code generation, released by Meta in August 2023, built by further training the Llama 2 foundation model on code-heavy data, and released as open weights in multiple sizes and…
Definition
Code Llama is a family of large language models specialized for code generation, released by Meta in August 2023, built by further training the Llama 2 foundation model on code-heavy data, and released as open weights in multiple sizes and variants.
Overview
Code Llama takes Meta's general-purpose Llama 2 model and continues training it on a large corpus weighted heavily toward source code, producing a model substantially stronger at code-related tasks than the base Llama 2 model while retaining much of its general language ability. It was released in multiple parameter sizes — 7B, 13B, and 34B, with a 70B variant added later — letting developers pick a size that fits their available hardware and latency requirements. The family includes several specialized variants: the base Code Llama models for general code completion, Code Llama - Python, further fine-tuned specifically on Python code for developers whose primary language is Python, and Code Llama - Instruct, tuned to follow natural-language coding instructions and produce safer, more helpful responses in a chat-style interface rather than only completing raw code. Code Llama also supports a notably long context window, useful for working with larger files and codebases. As an open-weight release, Code Llama became a popular foundation for building self-hosted coding assistants, IDE plugins, and fine-tuned internal tools where organizations wanted control over their code data rather than sending it to a third-party API. It's frequently compared with other open code models such as StarCoder, and both are commonly used within the Hugging Face ecosystem for further fine-tuning and deployment.
Key Concepts
- Built by further training Llama 2 on a large, code-heavy dataset
- Released by Meta in August 2023 as open weights
- Available in 7B, 13B, 34B, and later 70B parameter sizes
- Includes Python-specialized and Instruct-tuned variants
- Supports a long context window suited to larger files and codebases
- Popular base for self-hosted coding assistants and IDE integrations