NVIDIA TensorRT is a software development kit (SDK) and inference optimization runtime for deploying trained deep learning models on NVIDIA GPUs. It ingests models from ONNX, PyTorch and TensorFlow and compiles them into optimized runtime engines for low-latency, high-throughput inference.
Optimizations include INT8/FP8/FP16 quantization, layer and tensor fusion, and kernel auto-tuning. It exposes C++ and Python APIs and targets data-center GPUs, workstations and embedded platforms such as NVIDIA Jetson and DriveOS. The core SDK is proprietary, while open-source components (plugins, ONNX parser, samples) are published on GitHub under Apache 2.0. It was first released in 2017.
An SDK (Software Development Kit) is a curated set of libraries, interfaces, tools, sample code, and documentation intended for building applications and integrating with a specific hardware device, platform, or service. In robotics, an SDK typically exposes device control, telemetry, sensor access, configuration, and execution functions, significantly reducing the time-to-first-integration for developers targeting a specific robot or platform.
A Runtime is the environment or execution layer used to run code, load libraries, manage dependencies, and operate applications or services — either in real time or during normal system operation. In robotics this includes real-time operating system (RTOS) runtimes, ROS 2 executor runtimes, containerised execution environments (Docker, podman), and embedded C++ runtimes on microcontrollers.
A family of open-source operating systems based on the Linux kernel, common on servers, in the cloud and in robotics.
QNX Neutrino — the BlackBerry QNX microkernel RTOS with a POSIX API. Dominant in automotive ADAS / infotainment (235M+ vehicles) and safety-critical robotics.
Requires a compatible CUDA Toolkit version (CUDA 12.x line).
License family: Proprietary – Commercial
License family: Permissive
No version history.
