Encoder-Decoder Model
An encoder-decoder model is a neural network architecture split into two components — an encoder that processes an input sequence into an internal representation, and a decoder that generates an output sequence from that representation —…
Definition
An encoder-decoder model is a neural network architecture split into two components — an encoder that processes an input sequence into an internal representation, and a decoder that generates an output sequence from that representation — commonly used for tasks that map one sequence to another.
Overview
The encoder-decoder pattern, also called sequence-to-sequence modeling, addresses tasks where the input and output are both sequences but may differ in length and structure, such as translating a sentence from one language to another, summarizing a long document into a short one, or converting speech audio into text. The encoder reads the entire input sequence and compresses it into an internal representation — historically a single fixed-length vector in early recurrent implementations, and later a full sequence of contextualized representations in attention-based models. The decoder then generates the output sequence step by step, conditioning each generated token on both the encoder's representation and the tokens it has already produced. Early encoder-decoder models were built from recurrent neural networks like LSTMs, but these struggled with long input sequences because compressing an entire sentence into one fixed-length vector created an information bottleneck. The introduction of the attention mechanism let the decoder look back at different parts of the encoder's output at each generation step rather than relying solely on a single compressed summary, dramatically improving performance on long sequences. This insight directly motivated the Transformer architecture, which replaced recurrence entirely with self-attention and cross-attention, becoming the standard implementation of the encoder-decoder pattern. Models like the original T5 and BART use a full encoder-decoder Transformer, well suited to tasks with clear input-output transformations, such as translation and summarization. This contrasts with decoder-only architectures like GPT, which handle generation tasks using only a single stack conditioned on all prior tokens, without a separate encoding phase, and encoder-only models like BERT, which are used for understanding tasks without any generation. The choice between these architectural patterns depends on whether a task fundamentally involves transforming one sequence into a structurally different one (favoring encoder-decoder) or open-ended generation and understanding (favoring decoder-only or encoder-only designs).
Key Concepts
- Splits the model into a separate encoder and decoder component
- Encoder compresses the input sequence into an internal representation
- Decoder generates the output sequence conditioned on the encoder's output
- Well suited to tasks mapping one sequence to a structurally different one
- Early versions built on recurrent networks, later replaced by Transformers
- Attention mechanism lets the decoder reference all encoder outputs, not just a summary
- Used in models like T5 and BART for translation and summarization
- Contrasts with decoder-only (GPT) and encoder-only (BERT) architectures
Use Cases
Frequently Asked Questions
From the Blog
Reranking in RAG: When a Cross-Encoder Earns Its Latency
A cross-encoder reranker earns its latency when your first-stage retriever has high recall at a wide k but poor ordering in the top few. This article explains the two-stage pattern, the recall condition that makes reranking worthwhile, and how to measure whether it is paying for itself.
Read More AI & TechnologyImage Generation Systems: From Prompt to Pixels
A modern image generator is three components in a row: a text encoder, a denoiser working in latent space, and a decoder. Learn what each stage does, which setting affects which stage, and how to debug an image that came out wrong instead of rerolling the seed.
Read More AI & TechnologyReranking in RAG: cross-encoders, cost and how far to widen retrieval
Reranking lets you retrieve wide and send few passages. Learn candidate-set sizing, cross-encoder trade-offs, latency budgets and how to prove the gain.
Read More