AI Accelerator · serves as: AI acceleration, AI Inference, Compute, High-level compute.
Which group NVIDIA H200 belongs to and how it is built
Compute Modules is a subcategory of hardware components that provide processing power for robotic systems. It encompasses onboard computers, single-board computers (SBCs), AI accelerators, embedded processors, GPU/NPU compute modules, and other units responsible for processing sensor data and executing control logic. These modules form the foundation of modern autonomous, humanoid, and perception-capable robots.
An AI Accelerator is a specialized hardware component designed for efficient execution of artificial intelligence computations, particularly neural network inference, computer vision processing, and sensor data analysis. In robotics, AI accelerators are used to run perception models, object recognition, image segmentation, planning, and other tasks that require high computational throughput under constrained power budgets. They may take the form of dedicated NPU, TPU, VPU, or GPU chips, or specialized embedded modules.
A data-center AI accelerator card is a design class describing the construction of high-performance compute processors (GPUs/accelerators) intended for mounting in data-center servers. It is characterised by: an SXM form factor (a module soldered onto an HGX/DGX baseboard) or a dual-slot PCIe card; high-bandwidth memory (HBM2e/HBM3/HBM3e) integrated on-package; dedicated GPU-to-GPU interconnects (NVLink, Infinity Fabric) with hundreds of GB/s of bandwidth; high TDP (350–1000 W) requiring air or liquid cooling; support for virtualisation/partitioning (MIG) and low-precision compute formats (FP8/FP16/BF16/INT8). The class includes designs such as NVIDIA H100/H200/A100, AMD Instinct MI300, Google TPU, and Intel Gaudi. It describes physical construction and configuration, not the functional role (which is given by the component type "AI Accelerator").
NVIDIA H200 is a Hopper-generation data-center AI accelerator, unveiled on 13 November 2023 and available commercially from Q2 2024. It is the first GPU equipped with HBM3e memory: 141 GB at 4.8 TB/s of bandwidth — nearly double the capacity and 1.4× the memory bandwidth of the H100. It is based on the same Hopper-architecture die as the H100, so it offers identical compute and gains its advantage solely from larger, faster memory.
In the SXM variant, the H200 delivers up to 3,958 TFLOPS in FP8 and 1,979 TFLOPS in FP16/BF16 (with sparsity), at a TDP configurable up to 700 W — the same power profile as the H100. It retains 4th-generation Tensor Cores with FP8, the Transformer Engine, NVLink at 900 GB/s, PCIe Gen5 (128 GB/s), Multi-Instance GPU up to 7 instances of 18 GB each, and Confidential Computing. The H200 NVL variant is a dual-slot, air-cooled PCIe card with a TDP up to 600 W, intended for mainstream rack servers, supporting 2- or 4-way GPU linking via an NVLink bridge.
The larger memory makes the H200 especially efficient for large-language-model inference — NVIDIA cites up to 1.9× higher throughput in Llama 2 70B inference versus the H100 within the same power budget. The H200 is the memory-focused upgrade of the Hopper line, succeeding the H100, with the next architectural generation being Blackwell (B100/B200/GB200).
