AudioCraft
AudioCraft is an open-source generative audio framework released by Meta AI that bundles the MusicGen, AudioGen, and EnCodec models for music, sound effect, and general audio generation and compression.
Definition
AudioCraft is an open-source generative audio framework released by Meta AI that bundles the MusicGen, AudioGen, and EnCodec models for music, sound effect, and general audio generation and compression.
Overview
AudioCraft was released by Meta AI in 2023 as a unified PyTorch-based library for training and running state-of-the-art generative audio models, consolidating several of Meta's research efforts into a single, developer-friendly codebase. It bundles three main components: MusicGen, a text- (and optionally melody-) conditioned music generation model; AudioGen, a text-to-sound-effect model capable of generating environmental and non-musical audio like footsteps, rain, or applause; and EnCodec, a neural audio codec that compresses raw audio into discrete tokens, serving as the shared tokenization backbone that both MusicGen and AudioGen build on. The framework's design reflects a broader trend in generative audio research of treating audio generation as a discrete sequence modeling problem, similar to how large language models treat text — audio is first compressed into a sequence of discrete tokens by a codec like EnCodec, and then a transformer-based model learns to generate new token sequences autoregressively, which are decoded back into audio. By open-sourcing this full pipeline, including training code, inference code, and pretrained checkpoints, Meta made it substantially easier for researchers and developers to reproduce, extend, and fine-tune generative audio models without building the underlying tokenization and modeling infrastructure from scratch. AudioCraft has become a common reference implementation and starting point in academic and industry research on generative audio, cited widely in follow-up work on music generation, sound design tools, and audio codec research. Its permissive open release, combined with the practical usefulness of EnCodec as a general-purpose audio tokenizer, has made components of AudioCraft reusable well beyond Meta's original MusicGen and AudioGen models.
Key Features
- Unified PyTorch library bundling MusicGen, AudioGen, and EnCodec
- EnCodec provides shared neural audio tokenization/compression
- Treats audio generation as discrete sequence modeling, similar to text LLMs
- Open-source training and inference code with pretrained checkpoints
- Supports both music generation (MusicGen) and sound-effect generation (AudioGen)
- Widely used as a research reference implementation for generative audio