Edge Computing
How it works
Data is processed on the end device or on a nearby edge node (server, gateway, cloudlet), typically in a three-tier architecture: device — edge — cloud. As much computation as local resources allow is performed at the edge (filtering, aggregation, model inference), while only processed data or tasks requiring more power are sent to the cloud. This shortens response time and reduces the amount of data transmitted. In AI applications the model is compressed (quantization, distillation) so it fits within the constrained resources of the edge device and runs at acceptable latency.
Problem solved
The centralized cloud introduces latency, consumes network bandwidth and requires sending sensitive data to a remote data center. With the explosive growth of IoT devices and sensors, shipping all raw data to the cloud is costly and too slow for real-time applications.
Components
End device (sensor, camera, robot, phone) that generates data and often performs local computation.
Server, gateway or cloudlet near the devices that takes over part of the computation and data aggregation.
Centralized data center handling tasks beyond the edge capacity and model training.
Mechanisms deciding which tasks run on the device, at the edge, or in the cloud; managing model updates.
Implementation
AI models must be compressed (quantization, distillation) to fit the edge memory and power budget.
Data travels between many nodes, requiring cloud-independent encryption and a decentralized trust model.
Managing deployments and model updates across thousands of heterogeneous devices is operationally hard.
Evolution
First systems bringing content closer to users (e.g. Akamai) — the root of the edge idea.
M. Satyanarayanan et al. (Carnegie Mellon) introduce the cloudlet — a small edge data center for mobile applications.
Cisco popularizes fog computing as an extension of the cloud toward the edge for IoT.
ETSI establishes the Mobile Edge Computing group; Karim Arabi popularizes the term "edge computing" in an IEEE DAC 2014 keynote.
The first Jetson module with an embedded GPU enables on-device AI inference, key for robotics.
The paper by Shi et al. (IEEE IoT Journal) becomes the academic foundation for framing edge computing.
ETSI renames the standard from Mobile to Multi-access Edge Computing, extending it beyond cellular networks.
Compute bottleneck
Edge devices have limited compute, memory and power budget, which constrains the size and complexity of the models that can run on them.
Parallelism
The workload is distributed across many independent edge devices and nodes.
Hardware requirements
Edge computing is a deployment paradigm independent of any specific hardware architecture.
Embedded GPUs (e.g. NVIDIA Jetson) accelerate AI inference at the edge.
Edge TPU-class accelerators are designed for energy-efficient inference at the edge.
Neuromorphic chips offer very low power draw, attractive for edge devices.