Speech-to-Text Model
A speech-to-text (STT) model, also called automatic speech recognition (ASR), converts spoken audio into written text, transcribing what was said so it can be read, searched, or processed by downstream language models.
Definition
A speech-to-text (STT) model, also called automatic speech recognition (ASR), converts spoken audio into written text, transcribing what was said so it can be read, searched, or processed by downstream language models.
Overview
STT models take raw audio waveforms and output the corresponding transcript, handling the substantial variability of human speech — accents, background noise, overlapping speakers, and varying speaking speed. Modern systems, such as OpenAI's Whisper and similar Transformer-based models, are trained on large amounts of paired audio and transcript data, often spanning many languages, and can additionally perform tasks like translation from spoken audio directly into text in another language. Architecturally, most current STT models process audio in short overlapping windows, convert it into a spectrogram-like representation, and feed that through an encoder-decoder Transformer, similar in spirit to the architectures used for text generation, but adapted for continuous acoustic signals rather than discrete word tokens. Some systems add speaker diarization — identifying who spoke when — on top of raw transcription, one of several ways Multimodal Models bring non-text data into a language pipeline alongside a Foundation Model. STT is the input half of most voice-driven AI products: a user's speech is transcribed to text, that text is handed to a language model for understanding and response generation, and a Text-to-Speech Model can then speak the reply back. Beyond conversational AI, STT is widely used for meeting transcription, subtitle and captioning generation, voice search, and dictation tools, and it's a foundational building block for making audio and video content searchable and accessible.
Key Concepts
- Transcribes spoken audio into written text (automatic speech recognition)
- Handles variability in accents, noise, and speaking pace via large-scale training
- Modern systems are typically Transformer-based encoder-decoder models
- Many models support multiple languages and can translate speech directly to text
- Can be extended with speaker diarization to identify who is speaking
- Serves as the input stage for most voice-driven conversational AI systems
Use Cases
Frequently Asked Questions
From the Blog
Generative AI Explained: From Text to Images
Generative AI creates new content from patterns it learned — understand how text generation, image synthesis, and more work.
Read More AI & TechnologyMultimodal AI Explained: Text, Images, and Beyond
Multimodal AI processes and connects several data types like text, images, audio, and video all at once. Here is how it works and why it matters now.
Read More AI & TechnologyTokenization Explained: How LLMs Read Text
LLMs do not read words or letters; they read tokens, the chunks text is split into. Learn what tokens are and why they shape cost, limits, and behavior.
Read More AI & TechnologySpeech-to-Text With Whisper: A Practical Guide
Whisper turns spoken audio into accurate text across many languages. Learn how it works and how to transcribe your first audio file the practical way.
Read More