The Tesla FSD Neural Network Stack is a proprietary production neural network stack powering the Full Self-Driving (FSD) feature in Tesla cars. From FSD v12 (2024) onward it is an end-to-end "photons in, controls out" architecture — raw pixels from eight cameras feed directly into a policy that controls steering, throttle, and brake, with minimal hand-engineered code. The stack is also adapted by Tesla Bot (Optimus) for indoor navigation.
The architecture consists of several components: HydraNet — a multi-scale ResNet/RegNet backbone with a transformer head for object detection; Occupancy Networks — dense 3D occupancy prediction around the vehicle (volumetric voxel grid 200×200×16); Lane Network — an auto-regressive model for road lines and intersection topology; Neural Planner — a policy network generating trajectories. Everything is trained on a Dojo + H100 GPU cluster with ~10M video clips from the fleet (auto-labeling).
The stack is fully closed-source and runs exclusively on Tesla Hardware 3 (FSD Chip, 144 TOPS) and Hardware 4 (~7× faster). Full-precision real-time inference across 8 camera channels at 36 FPS. The Optimus version is an adaptation of the same runtime, with different output modalities (joint torques instead of steering/throttle).
A Runtime is the environment or execution layer used to run code, load libraries, manage dependencies, and operate applications or services — either in real time or during normal system operation. In robotics this includes real-time operating system (RTOS) runtimes, ROS 2 executor runtimes, containerised execution environments (Docker, podman), and embedded C++ runtimes on microcontrollers.
A Perception Stack encompasses the software layers that process data from cameras, LiDARs, IMUs, microphones, and other sensors in order to recognise the surrounding environment, perform localisation, detect and track objects, and interpret the scene. It is typically the first processing stage in an autonomous robot's data pipeline, feeding its outputs to planning and control stacks.
A Control Stack is the set of software components responsible for control logic, motion planning, command execution, and coordination of a robot's actuators. It receives high-level goals or waypoints (from planning or AI layers) and translates them into low-level actuator commands, closed-loop feedback control, and safety checks. Common implementations include MoveIt (arm manipulation), Nav2 (mobile navigation), and custom whole-body control stacks for humanoids.
~3M Tesla cars with FSD on public roads (2025). Tesla Optimus Gen 2/3 (several thousand units in Tesla factories, public demos).
No open-source community. All communication via Tesla AI Day, Twitter/X (@elonmusk, @aelluswamy), and the corporate blog.
Locked to Tesla products. The Optimus version is adapted to Jetson Thor.
License family: Proprietary – Commercial
5× more parameters, HW4 native, −30% latency.
FSD adapted for Tesla Optimus humanoid.
End-to-end "photons to controls" architecture.
Unified stack for highway and city streets.
Introduction of Occupancy Networks.
