Nemotron-H-8B-128K

Nemotron-H-8B-128K is an approximately 8-billion-parameter large language model (LLM) developed by NVIDIA as part of the Nemotron-H family. It uses a hybrid architecture consisting primarily of Mamba-2 (state space model) and MLP layers, combined with just four attention (Transformer) layers. The design aims to deliver accuracy comparable to pure Transformer models while lowering inference cost, particularly for long context.

The model supports a context window of up to 128,000 tokens and operates on text input and output. It supports ten languages: English, German, Spanish, French, Italian, Korean, Portuguese, Russian, Japanese and Chinese. Its pretraining data has a cutoff of September 2024.

The model is released in Base, Instruct and Reasoning (128K) variants, as well as BF16 and FP8 checkpoints. The Reasoning variant can generate a reasoning trace before its final answer, and reasoning mode can be toggled via the system prompt. The model is intended for research and development only and is governed by the NVIDIA Internal Scientific Research and Development Model License.

Its architecture, training and evaluation are described in the technical report "Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models" (arXiv:2504.03624).