ONNX Runtime is an open-source, cross-platform accelerator for inference and training of machine-learning models stored in the ONNX (Open Neural Network Exchange) format. The project is developed and maintained by Microsoft and released under the MIT license.
Its core is written in C++, with official APIs available for Python, C#/.NET, Java and JavaScript, among others. The engine runs on Linux, Windows, macOS, iOS and Android as well as in the browser, across environments from cloud to edge and mobile devices.
Performance comes from pluggable execution providers that leverage hardware acceleration: CPU, NVIDIA CUDA and TensorRT, DirectML (Windows), Apple CoreML, Intel OpenVINO, Qualcomm QNN and others. ONNX Runtime powers Microsoft products including Windows, Office, Azure services and Bing.
An inference runtime loads a pre-trained model and runs forward passes (predictions) on input data. It typically supports hardware acceleration (CPU, GPU, NPU) through pluggable execution backends.
A software library exposing APIs to build, deploy or execute machine-learning models within applications.
Used in Windows, Microsoft Office, Azure services and Bing.
Over 21,000 GitHub stars and ~4.2k forks (microsoft/onnxruntime).
A compiled, high-performance, general-purpose programming language. The primary implementation language of the ONNX Runtime core.
A high-level interpreted language dominant in machine learning. ONNX Runtime provides an official Python API.
An object-oriented programming language by Microsoft for the .NET platform. ONNX Runtime provides an official C#/.NET API.
A family of open-source operating systems based on the Linux kernel, common on servers, in the cloud and in robotics.
Microsoft's family of operating systems for personal computers and servers.
Apple's operating system for Mac computers.
Apple's mobile operating system for iPhone and iPad devices.
A mobile operating system based on the Linux kernel, developed by Google.
The Python package manager. The onnxruntime package is distributed via PyPI.
The package manager for the .NET platform. ONNX Runtime ships as Microsoft.ML.OnnxRuntime packages.
The package manager for the JavaScript/Node.js ecosystem. The onnxruntime-web and onnxruntime-node packages are distributed via npm.
A dependency and build manager for the Java ecosystem. ONNX Runtime is distributed as com.microsoft.onnxruntime.
A 64-bit CPU architecture compatible with the x86 instruction set, dominant in PCs and servers.
A 64-bit ARM CPU architecture, common in mobile, embedded and increasingly server devices.
Deployment in a cloud environment (servers, containers, managed services).
Local deployment on an edge or embedded device, without cloud dependency.
Deployment on mobile devices (iOS, Android).
Execution in a web browser (e.g. via WebAssembly / WebGPU).
A permissive free-software license originating at the Massachusetts Institute of Technology. It permits use, copying, modification, merging, publishing, distribution, sublicensing and sale of the software, provided the copyright notice and license text are preserved. It has no copyleft.
A permissive license fully compatible with the ROS/ROS 2 ecosystem and with closed-source commercial products.
Latest stable release in the 1.x series.
First stable 1.0 release (general availability).
First public release of ONNX Runtime as an open-source project.
