CED
How it works
The model is divided into an encoder and a decoder within a single causal Transformer with Mixture-of-Experts layers. During the prefill phase the entire input context is processed (in DeepSeek-V4.1-Flash about 8B parameters are active per token), and during the decode phase, when generating the response, a larger number of parameters is activated (about 16B). This reserves the more expensive computation for the actual generation, and the cheaper computation for ingesting the long context.
Problem solved
Classic decoder-only models activate the same large number of parameters both when processing a long context and when generating each token, which is costly. CED lets the compute cost be matched to the model's operating phase.
Evolution
The Causal Encoder-Decoder architecture introduced in DeepSeek-V4 models (including V4.1-Flash).