The model is divided into an encoder and a decoder within a single causal Transformer with Mixture-of-Experts layers. During the prefill phase the entire input context is processed (in DeepSeek-V4.1-Flash about 8B parameters are active per token), and during the decode phase, when generating the response, a larger number of parameters is activated (about 16B). This reserves the more expensive computation for the actual generation, and the cheaper computation for ingesting the long context.
Classic decoder-only models activate the same large number of parameters both when processing a long context and when generating each token, which is costly. CED lets the compute cost be matched to the model's operating phase.
The Causal Encoder-Decoder architecture introduced in DeepSeek-V4 models (including V4.1-Flash).
