Robots Atlas>ROBOTS ATLAS
Architecture

TDT

2023ActivePublished: 16 October 2026Updated: 16 October 2026Published
Key innovation
Extends the Transducer (RNN-T) architecture by jointly predicting a token and its duration (the number of input frames it covers), letting the decoder skip frames and run inference much faster than a conventional Transducer that processes the encoder output frame by frame.
Category
Architecture
Abstraction level
Building block
Operation level
Architecture blockInferenceModel
Use cases
Automatic speech recognition (ASR)Speech translationSpeech intent classification and slot fillingLow-latency speech model inference in production

How it works

TDT's joint network has two outputs: one produces a distribution over tokens (including the blank symbol), the other a distribution over a predefined set of possible durations (the number of frames to skip). The two outputs are normalized independently. At each decoding step the model picks a token and a duration; the predicted duration determines how many encoder frames to advance before the next step. This skips frames that carry no new tokens and reduces the number of decoding steps. The model is trained with a variant of the transducer loss that accounts for duration prediction.

Problem solved

During decoding a conventional Transducer (RNN-T) processes the encoder output frame by frame and spends many steps emitting the blank symbol without producing a token, which slows inference. TDT reduces this overhead by letting the model skip non-informative frames based on the predicted duration.

Components

Acoustic encoderTransforms the input signal into a sequence of frame representations.

The encoder (e.g. Conformer/FastConformer in Parakeet models) maps audio features to hidden frame representations.

Prediction networkAn autoregressive label decoder modelling the history of emitted tokens.

The RNN-T-style decoder; provides language context conditioned on previously emitted tokens.

Joint network with two outputsCombines encoder and prediction-network representations and produces two independently normalized distributions.

Unlike a conventional Transducer it has two heads: a distribution over tokens and a distribution over durations.

Implementation

Reference implementations
NVIDIA NeMo
Python · NVIDIA
Official

Evolution

Original paper · 2023 · Hainan Xu
Efficient Sequence Transduction by Jointly Predicting Tokens and Durations
Hainan Xu, Fei Jia, Somshubra Majumdar, He Huang, Shinji Watanabe, Boris Ginsburg
2023
Introduction of the TDT architecture
Inflection point

Xu et al. publish TDT as an RNN-T extension predicting a token and its duration; the implementation lands in NVIDIA NeMo.

Efficient Sequence Transduction by Jointly Predicting Tokens and Durations (paper)
2024
Adoption in NVIDIA Parakeet models

TDT becomes the decoder in the Parakeet model family (e.g. parakeet-tdt-1.1b), pairing a FastConformer encoder with a TDT decoder.

Hyperparameters (configurable axes)

Duration value setHigh

The predefined set of duration values (number of frames to skip) the model can predict. A wider range enables more aggressive frame skipping and faster inference, potentially at the cost of accuracy.

Execution paradigm

Primary mode
Dense

The network is dense (all parameters active). The dynamics come from conditionally skipping input frames during decoding based on the predicted duration.

Activation pattern
All paths active

Parallelism

Parallelism level
Sequential

Decoding is autoregressive (sequential), but frame skipping reduces the number of steps versus a conventional Transducer; training is parallelized.

Scope
Training