Architecture

Seq2Seq RNN

2014HistoricalPublished: 28 May 2026Updated: 28 May 2026Published

Key innovation

Framing sequence transduction as two jointly trained RNNs: an encoder that compresses the input into a fixed-length vector and a decoder that generates the output sequence.

Category

Architecture

Abstraction level

Pattern

Operation level

ModelTrainingInference

Use cases

Machine translationSequence transductionText summarizationSpeech recognition

How it works

An RNN encoder reads input tokens one by one and updates its hidden state. The final encoder state is used as a context vector representing the whole sequence. An RNN decoder starts from that vector and autoregressively generates output tokens, maximising the probability of the target sequence conditioned on the input.

Problem solved

It models tasks where both input and output are variable-length sequences without manually engineering alignments between sequence elements.

Components

RNN encoderInput encoding.

Recurrent network that reads the input sequence and produces a context representation.

Official

Fixed-length context vectorBridge between encoder and decoder.

Final encoder state used as the representation of the whole input sequence.

RNN decoderOutput decoding.

Recurrent network that generates the output sequence autoregressively.

Official

Evolution

Original paper · 2014 · EMNLP 2014 · Kyunghyun Cho

Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio

Computational complexity

Time complexity: O(T_x · d² + T_y · d²).

Execution paradigm

Primary mode

Dense

Activation pattern

All paths active

Parallelism

Parallelism level

Sequential

RNNs process tokens sequentially along the time dimension.

Scope

TrainingInference

Sources

Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation