AWS Trainium
AWS Trainium is a family of custom AI accelerators (ASICs) designed by Amazon Web Services with one goal: the best economics for high-performance AI training and inference at scale. Trainium sits at the center of a fully integrated system - chip, server, network, software and services - co-designed to work as one, optimizing cost-per-token at production scale.
The chip is built around NeuronCore cores. The latest generation, Trainium3, features the 4th-generation NeuronCore with eight large cores, each with four specialized engines (Tensor, Vector, Scalar, GPSIMD) running simultaneously, offering up to 2x higher MXFP8 compute throughput versus Trainium2. Each chip has 144 GB of HBM3e memory at 4.9 TB/s bandwidth (1.7x higher than Trainium2) and a two-level on-chip SRAM hierarchy.
Trainium scales from a single chip to millions: it interconnects via NeuronLink (with redundancy for fault tolerance) and Elastic Fabric Adapter (EFA), with Graviton as the host CPU and Nitro for security. It is supported by the Neuron SDK, Neuron Kernel Interface (NKI, ISA access), PyTorch/JAX, Amazon EKS and SageMaker HyperPod. It is available in EC2 (Trn) instances and UltraServer/UltraCluster configurations. The line is complemented by AWS Inferentia (inference).
Classification
Custom AI training and inference ASIC
Which group AWS Trainium belongs to and how it is built
Covers Application-Specific Integrated Circuit (ASIC) chips designed inside hyperscalers (Google, Amazon, Meta, Microsoft) in partnership with semiconductor partners (Broadcom, Marvell, Alchip). Properties: modular chiplet architecture (compute, network, I/O as separate silicon blocks), HBM integration, support for low precisions (MX8/MX4, FP8), dedicated ML engines (Dot Product Engine, Attention Engine, Reduction Engine), PyTorch-native software stack. Examples: Google TPU, AWS Trainium/Inferentia, Meta MTIA, Microsoft Maia.
A data-center AI accelerator card is a design class describing the construction of high-performance compute processors (GPUs/accelerators) intended for mounting in data-center servers. It is characterised by: an SXM form factor (a module soldered onto an HGX/DGX baseboard) or a dual-slot PCIe card; high-bandwidth memory (HBM2e/HBM3/HBM3e) integrated on-package; dedicated GPU-to-GPU interconnects (NVLink, Infinity Fabric) with hundreds of GB/s of bandwidth; high TDP (350โ1000 W) requiring air or liquid cooling; support for virtualisation/partitioning (MIG) and low-precision compute formats (FP8/FP16/BF16/INT8). The class includes designs such as NVIDIA H100/H200/A100, AMD Instinct MI300, Google TPU, and Intel Gaudi. It describes physical construction and configuration, not the functional role (which is given by the component type "AI Accelerator").
Technical specification
Basic physical properties of AWS Trainium โ dimensions, weight and materials
Relations
Other hardware parts related to AWS Trainium
A data-center AI accelerator based on the NVIDIA Hopper architecture (2022): 80 GB HBM3, 700 W, 4th-generation Tensor Cores with FP8 and Transformer Engine. The standard GPU for LLM training and hyperscale AI inference.
A data-center AI accelerator based on the NVIDIA Hopper architecture (2023): the first GPU with HBM3e memory โ 141 GB at 4.8 TB/s. Same compute as the H100 but nearly double the memory capacity and bandwidth for training and inference of large language models.