Whisper
By OpenAI
Whisper is an open-source automatic speech recognition (ASR) model from OpenAI that transcribes and translates spoken audio into text across dozens of languages.
Definition
Whisper is an open-source automatic speech recognition (ASR) model from OpenAI that transcribes and translates spoken audio into text across dozens of languages.
Overview
Whisper was released by OpenAI in 2022 as an open-source speech-to-text model trained on a large, diverse dataset of multilingual and multitask audio collected from the web. Unlike many earlier ASR systems tuned narrowly for clean, single-language audio, Whisper was trained to handle accents, background noise, and technical language, and it can perform both transcription (speech to text in the same language) and translation (speech to English text) in a single model. Architecturally, Whisper uses an encoder-decoder Transformer, similar in spirit to the architecture underlying text-based LLMs, but adapted to process audio spectrograms as input rather than text tokens. OpenAI released several model sizes, from a small, fast 'tiny' variant suited to on-device or real-time use, up to a 'large' variant optimized for maximum accuracy. Because Whisper's weights are openly available, it has been widely adopted well beyond OpenAI's own products — used in voice assistants, meeting-transcription tools, subtitle generation, accessibility software, and as a component of larger voice-enabled AI agents.
Key Features
- Open-source weights, freely downloadable and self-hostable
- Supports transcription and translation across dozens of languages
- Multiple model sizes trading off speed against accuracy
- Encoder-decoder Transformer architecture adapted for audio input
- Robust to accents, background noise, and technical vocabulary
- Widely integrated into third-party voice and transcription tools