TOPS is derived from an accelerator's hardware specification, not from a measurement. The basic formula is: TOPS = (number of MAC units × clock frequency × 2) / 10^12, where the factor of 2 reflects that each MAC operation comprises one multiply and one add. Adding more MAC units or higher clocks raises TOPS. Because compute units process lower-precision data faster (e.g. twice as many INT4 as INT8 operations per cycle), the TOPS value increases as precision drops. Structured sparsity techniques let the chip skip zero-valued weights, so vendors additionally report 'sparse TOPS'. Real inference performance, however, depends not only on TOPS but also on memory bandwidth, model size, and utilization efficiency, which is why peak TOPS is an upper bound rather than a guarantee.
The AI accelerator market needed a single, easy-to-communicate number describing an inference chip's raw compute power, to compare hardware and estimate whether it can run a given model. TOPS fills that role, but also exposes a problem: without stating the precision (INT8/INT4/FP) and conditions (sparsity), a single number can be misleading and fails to reflect real-world performance.
