NEON
How it works
NEON packs multiple data elements (e.g. sixteen 8-bit or four 32-bit values) into a single 128-bit vector register and applies one instruction to all of them at once (SIMD). The instruction set covers integer and floating-point arithmetic and logic, shifts, saturating operations, vector load/store and multiply-accumulate instructions, including fused multiply-add (FMA, from VFPv4/Armv8), which is fundamental to GEMM and convolution kernels in ML. Compilers emit NEON via auto-vectorisation, or developers use intrinsics/assembly; libraries (e.g. Ne10, Arm Compute Library, llama.cpp kernels) ship ready, optimised routines. On AArch64 NEON uses 32 128-bit V registers and provides IEEE 754-compliant rounding.
Problem solved
Scalar CPU instructions process one value at a time, which is inefficient for regular, bulk computation over vectors and matrices (multimedia, DSP, neural-network multiply-accumulate). Separate coprocessors/DSP cores complicated programming and portability. NEON provides a standardised vector instruction set built into the ARM CPU, letting many operations run in parallel without a dedicated accelerator.
Components
In AArch32: 16 128-bit Q registers (Q0–Q15) overlapping 32 64-bit D registers (D0–D31). In AArch64: 32 independent 128-bit V registers (V0–V31). Shared with the VFP unit.
SIMD operations on 8/16/32/64-bit integers and single-precision (32-bit) floating point; AArch64 also adds double precision, IEEE 754-compliant. Cover arithmetic, logic, shifts and saturating operations.
Multiply-accumulate instructions performed in one step with a single rounding (from VFPv4/Armv8). Critical for the GEMM and convolution kernels used in ML inference.
Implementation
Evolution
The Armv6 architecture introduced early SIMD-for-multimedia instructions — the direct precursor of Advanced SIMD.
NEON appeared as a 64-/128-bit SIMD extension of the Armv7-A architecture; present in all Cortex-A8 cores, optional in Cortex-A9.
In Armv8-A, Advanced SIMD (NEON) became standard for both AArch32 and AArch64. AArch64 provides 32 independent 128-bit V registers and fully IEEE 754-compliant arithmetic, including double precision.
The Armv9-A architecture mainstreams SVE2 (Scalable Vector Extension 2) — a vector extension with scalable vector length, developed alongside NEON.
Execution paradigm
SIMD applies the same operation densely across all active vector lanes.
Parallelism
Data parallelism within a single vector instruction.
Hardware requirements
NEON is the native SIMD extension of ARM/AArch64 CPUs (Cortex-A, Apple Silicon, ARM servers, mobile and edge SoCs). A "CPU SIMD" category — the ARM counterpart of x86 AVX.