SAM 2
SAM 2 (Segment Anything Model 2) is Meta AI's 2024 successor to the original Segment Anything Model, extending promptable, zero-shot segmentation from still images to video, including the ability to track objects across frames.
Definition
SAM 2 (Segment Anything Model 2) is Meta AI's 2024 successor to the original Segment Anything Model, extending promptable, zero-shot segmentation from still images to video, including the ability to track objects across frames.
Overview
SAM 2 was released by Meta AI in July 2024 as a unified model for promptable segmentation across both images and video, building directly on the original SAM's architecture and design philosophy. Where SAM could segment objects in a single image given a prompt like a point or bounding box, SAM 2 extends this capability temporally: given a prompt on any single frame of a video, it can segment the target object and then track that segmentation mask consistently across subsequent frames, even through occlusion, motion, and changes in appearance. SAM 2's architecture introduces a streaming memory mechanism — a memory encoder, memory bank, and memory attention module — that lets the model retain information about previously segmented objects as it processes a video frame by frame, rather than treating each frame independently. This allows it to maintain object identity over time and to be corrected interactively: a user can add refining prompts on later frames, and the model propagates that correction both forward and backward through the video. SAM 2 was also trained on a substantially larger dataset than the original SAM's SA-1B, including SA-V, a new video segmentation dataset Meta created and released alongside the model, containing far more mask annotations across diverse video content. Because it unifies image and video segmentation in a single model, and does so with real-time performance suitable for interactive applications, SAM 2 has been adopted for video editing tools, augmented and mixed reality applications, scientific video analysis (such as tracking objects or organisms across footage), and autonomous systems research. Like its predecessor, Meta released SAM 2's weights, code, and the SA-V dataset openly, continuing the Segment Anything project's role as a widely used foundation for segmentation research and applications.
Key Concepts
- Unified promptable segmentation for both images and video
- Streaming memory mechanism for tracking objects across video frames
- Handles occlusion and appearance changes during object tracking
- Supports interactive correction that propagates forward and backward through a video
- Trained on the newly released SA-V video segmentation dataset
- Real-time performance suitable for interactive editing tools
- Open weights and code released by Meta AI