Sequence-to-Sequence Model
A sequence-to-sequence (seq2seq) model is a class of neural network models designed to transform an input sequence into an output sequence, where the two sequences can have different lengths, commonly implemented using an encoder-decoder…
Definition
A sequence-to-sequence (seq2seq) model is a class of neural network models designed to transform an input sequence into an output sequence, where the two sequences can have different lengths, commonly implemented using an encoder-decoder architecture.
Overview
Sequence-to-sequence modeling was introduced to solve tasks where both the input and desired output are sequences of variable, and often different, length — a structure that doesn't fit neatly into traditional classification or regression frameworks. The canonical example is machine translation, where an input sentence in one language must be mapped to an output sentence in another language that may have a completely different number of words and word order. Other examples include summarization (long document to short summary), speech recognition (audio signal to text transcript), and conversational response generation. The term seq2seq is most closely associated with the specific 2014 architecture proposed by Sutskever, Vinyals, and Le, which used two LSTM networks — an encoder to read the input sequence into a fixed-length vector and a decoder to generate the output sequence from that vector. This was quickly improved by adding an attention mechanism, letting the decoder reference the encoder's full sequence of hidden states rather than a single compressed summary, which substantially improved translation quality, particularly for longer sentences. Modern sequence-to-sequence tasks are almost universally handled using Transformer-based encoder-decoder models like T5 and BART, or by prompting general-purpose large language models to perform the transformation directly through instructions, since a sufficiently capable LLM can perform many traditionally seq2seq tasks like translation or summarization within a single decoder-only model, without a distinct encoder-decoder split. The term "sequence-to-sequence" is now used more broadly to describe the general problem category — mapping one sequence to another — rather than referring exclusively to the original RNN-based architecture, which has been largely superseded by attention-based approaches.
Key Concepts
- Maps an input sequence to an output sequence of potentially different length
- Originally implemented with paired encoder and decoder LSTM networks
- Introduced by Sutskever, Vinyals, and Le in 2014
- Improved substantially by adding attention mechanisms
- Modern implementations use Transformer-based encoder-decoder architectures
- Also solvable via prompting general-purpose decoder-only LLMs
- Term now describes a general problem category, not one specific architecture
- Core paradigm behind translation, summarization, and speech recognition
Use Cases
Frequently Asked Questions
From the Blog
The 2026 AI Engineer Roadmap: Skills, Tools, and Career Path
AI Engineer is one of the fastest-growing roles in tech — and it's more accessible than traditional ML engineering. This guide maps the exact skills, tools, and learning sequence for becoming an AI engineer in 2026, from Python basics to deploying production RAG and agent systems.
Read More AI & TechnologyWhat Is Prompt Chaining and When to Use It
Prompt chaining splits a big task into a sequence of smaller LLM calls, where each step's output feeds the next. Learn how it works and when to use it.
Read More AI & TechnologyWhat Is Self-Attention in Neural Networks
Self-attention lets each token in a sequence attend to every other token in the same sequence, building context-aware representations that power transformer models.
Read More AI & TechnologyWhat Is Ensemble Learning: Bagging and Boosting
Ensemble learning combines many models into one stronger predictor. Bagging trains them in parallel to cut variance; boosting trains them in sequence to cut bias.
Read More