An RNN encoder reads input tokens one by one and updates its hidden state. The final encoder state is used as a context vector representing the whole sequence. An RNN decoder starts from that vector and autoregressively generates output tokens, maximising the probability of the target sequence conditioned on the input.
It models tasks where both input and output are variable-length sequences without manually engineering alignments between sequence elements.
Recurrent network that reads the input sequence and produces a context representation.
Official
Final encoder state used as the representation of the whole input sequence.
Recurrent network that generates the output sequence autoregressively.
Official
Time complexity: O(T_x · d² + T_y · d²).
RNNs process tokens sequentially along the time dimension.
