Edge Computing for Robotics
NVIDIA Jetson, Real-Time AI & Fog Architecture

1. Executive Summary

Edge computing has become the foundational enabler of modern autonomous robotics. By processing sensor data, running AI inference, and executing control loops directly on the robot or at a nearby fog node, edge architectures eliminate the latency, bandwidth, and reliability constraints that make cloud-only approaches untenable for real-time robotic systems. The global edge computing market for robotics is projected to reach $9.8 billion by 2028, growing at a CAGR of 23.4% as manufacturers recognize that sub-10ms inference latency is not optional for safety-critical autonomy.

This guide provides a comprehensive technical framework for designing, selecting, and deploying edge computing infrastructure across the full spectrum of robotic systems -- from warehouse AMRs and manufacturing cobots to agricultural drones and last-mile delivery robots. We cover the leading hardware platforms (NVIDIA Jetson, Intel Movidius, Google Coral, Hailo-8, Qualcomm RB5), inference optimization frameworks (TensorRT, ONNX Runtime, OpenVINO), fog computing architectures, containerized deployment with K3s, and the emerging 5G-edge convergence that is unlocking new classes of mobile robotic applications.

Based on our deployment experience across 35+ edge-enabled robotic systems in the APAC region, properly architected edge solutions deliver 10-50x latency reduction over cloud inference, 80-95% bandwidth savings, complete data sovereignty compliance, and the deterministic real-time performance required for ISO 13849 safety-rated applications.

2. Edge Computing in Robotics Market

2.1 Market Growth Drivers

The convergence of three technology waves is driving explosive growth in edge computing for robotics. First, the rapid advancement of system-on-module (SoM) platforms has placed datacenter-class AI inference capability into 15-30W power envelopes that fit on mobile robots. NVIDIA's Jetson Orin family delivers up to 275 TOPS of INT8 inference performance -- equivalent to what required a full GPU server just five years ago. Second, the maturation of inference optimization toolchains (TensorRT, ONNX Runtime, OpenVINO) has made it practical to deploy production neural networks on constrained hardware without sacrificing accuracy. Third, 5G and Wi-Fi 6E networks are creating the reliable, low-latency connectivity needed for edge-cloud hybrid architectures where the edge handles time-critical inference and the cloud handles training, fleet analytics, and long-horizon planning.

Industry analysts estimate that by 2027, over 70% of robotic AI inference will occur at the edge rather than in the cloud, a dramatic inversion from the cloud-centric architectures that dominated the 2018-2022 era. This shift is accelerated by data privacy regulations (Vietnam's PDPD, GDPR, China's PIPL) that restrict transmission of camera feeds and sensor data to external cloud environments.

2.2 Market Segmentation by Robot Type

3. Why Edge Matters: Latency, Bandwidth & Privacy

3.1 Latency: The Physics of Real-Time Autonomy

For a robot operating at 1.5 m/s -- typical for a warehouse AMR -- every 100ms of decision latency translates to 15cm of travel distance before the robot can react to a detected obstacle. In safety-critical scenarios involving human workers, this gap between perception and action can mean the difference between a graceful stop and a collision. Cloud-based inference introduces 100-300ms of round-trip latency even on well-provisioned networks (network transit + serialization + queue time + inference + response), making it fundamentally unsuitable for real-time control loops.

Edge inference eliminates network transit entirely. A NVIDIA Jetson Orin NX running a YOLOv8 object detection model delivers inference results in 6-12ms, providing the deterministic sub-frame latency required for 30Hz control loops. For safety-rated applications under ISO 13849 or IEC 62443, edge processing is not merely preferred -- it is a compliance requirement, as safety functions cannot depend on network availability.

3.2 Bandwidth: Economics of Sensor Data

A single robot equipped with modern sensor arrays generates enormous data volumes. A typical AMR with two LiDAR sensors, four cameras, and an IMU produces 150-400 Mbps of raw sensor data. For a fleet of 50 robots, this translates to 7.5-20 Gbps of continuous upstream bandwidth -- a prohibitive requirement for any facility network, let alone a cellular connection. Edge processing reduces this to a trickle of structured telemetry (robot state, task updates, anomaly alerts) at 50-200 Kbps per robot, a 1000x bandwidth reduction.

3.3 Data Privacy & Sovereignty

Edge computing addresses critical data sovereignty requirements that are increasingly enforced across APAC. Vietnam's Personal Data Protection Decree (PDPD, effective 2023) restricts cross-border transfer of personal data, which includes camera feeds containing identifiable individuals. Robots operating in warehouses, hospitals, and public spaces continuously capture imagery of workers and civilians. Edge processing allows all privacy-sensitive inference (person detection, face blurring, behavior analysis) to occur locally, with only anonymized metadata transmitted to the cloud. This architecture satisfies PDPD compliance, Singapore's PDPA, and South Korea's PIPA without requiring expensive data localization infrastructure for raw sensor storage.

4. NVIDIA Jetson Platform Deep Dive

4.1 Jetson Orin Family Overview

The NVIDIA Jetson platform has established itself as the de facto standard for edge AI in robotics, commanding over 60% market share in autonomous machine applications. The Jetson Orin generation, built on the Ampere GPU architecture with dedicated DLA (Deep Learning Accelerator) cores, represents a generational leap in performance-per-watt that has fundamentally changed what is possible at the edge.

4.2 Jetson Orin NX: The Sweet Spot for Robotics

The Jetson Orin NX 16GB has emerged as the most widely deployed edge compute module in commercial robotics. At 100 TOPS within a 10-25W envelope, it supports simultaneous execution of multiple neural networks -- object detection, semantic segmentation, depth estimation, and path planning -- while leaving sufficient CPU headroom for ROS 2 middleware, fleet communication, and sensor drivers. Its 16GB unified memory eliminates the host-device memory copy overhead that plagues discrete GPU architectures, enabling zero-copy camera-to-inference pipelines.

4.3 Jetson AGX Orin: Multi-Sensor Autonomy

The AGX Orin targets premium robotic platforms requiring concurrent processing of 8-16 camera streams alongside LiDAR point clouds and radar data. Its 275 TOPS performance enables running transformer-based perception models (BEVFormer, StreamPETR) that were previously cloud-only. With 64GB of unified LPDDR5 memory, the AGX Orin can maintain large environmental maps, cache multiple model variants for different operating contexts, and run on-device model fine-tuning for continual learning. Power modes from 15W (standby navigation) to 60W (full autonomous operation) allow dynamic power management based on situational complexity.

4.4 NVIDIA Thor: The Next Frontier

Announced for 2026 availability, NVIDIA Thor represents a convergence of autonomous vehicle and robotics computing. Built on the Blackwell GPU architecture paired with a Grace ARM CPU, Thor delivers 800+ TOPS in a module form factor suitable for humanoid robots, advanced manufacturing cells, and L4 autonomous mobile platforms. Thor's most significant architectural innovation is its support for multi-domain safety isolation, enabling a single module to run safety-critical perception (ASIL-D rated), general autonomy, and infotainment workloads in hardware-isolated partitions. For robotics integrators, this eliminates the need for separate safety and compute boards, reducing BOM cost and system complexity.

5. Edge Hardware Ecosystem

5.1 Intel Edge Platforms

Intel's edge computing portfolio for robotics centers on two product lines: the Movidius VPU (now integrated as the Intel AI Boost NPU in Core Ultra processors) and the industrial-grade NUC/Edge Controller platforms. The Intel NUC 13 Pro, combined with an Arc A-series discrete GPU or OpenVINO-optimized models on the integrated NPU, provides a competitive alternative for robotics workloads where CUDA dependency is undesirable.

Intel's primary strength lies in its OpenVINO optimization ecosystem, which provides seamless model conversion from PyTorch, TensorFlow, and ONNX formats with automatic hardware-specific optimization. For enterprises already invested in Intel infrastructure, the x86 compatibility simplifies deployment of existing perception software without the ARM porting effort required by Jetson.

5.2 Google Coral Edge TPU

The Google Coral platform, powered by the Edge TPU ASIC, targets the cost-sensitive and ultra-low-power segment of the robotics edge market. The Coral M.2 Accelerator delivers 4 TOPS of INT8 inference at just 2W, making it ideal for battery-powered robots, agricultural drones, and distributed sensor nodes where power budget is the primary constraint. The Coral Dev Board (with NXP i.MX 8M SoC + Edge TPU) offers a complete Linux-based development platform at under $150.

Limitations include the Edge TPU's fixed INT8 quantization requirement and limited operator support -- complex models with custom layers may require significant modification to compile for the Edge TPU. Best suited for well-defined, single-model inference tasks rather than multi-model perception pipelines.

5.3 Hailo-8 AI Accelerator

The Hailo-8 processor has emerged as a compelling alternative for embedded robotics, delivering 26 TOPS at 2.5W in an M.2 module form factor. Its unique dataflow architecture avoids the memory bandwidth bottleneck that limits conventional neural network accelerators, achieving near-theoretical throughput utilization on architectures including ResNet, YOLO, SSD, and EfficientNet. Hailo-8 is deployed in production by leading AMR manufacturers including MiR (Mobile Industrial Robots) and Locus Robotics for real-time obstacle classification.

The Hailo-15 vision processor, released in late 2025, integrates the AI accelerator with ISP, video encoder, and ARM CPU into a single SoC -- eliminating the need for a separate host processor for simpler robotic applications. At under $50 in volume, Hailo-15 is positioned to disrupt the cost structure of intelligent robotic peripherals like smart grippers and vision-guided end effectors.

5.4 Qualcomm RB5 / RB6

Qualcomm's Robotics RB5 platform, based on the QCS8250 SoC, offers 15 TOPS AI performance combined with native 5G modem integration -- a unique advantage for mobile robots operating in outdoor or multi-site environments. The heterogeneous compute architecture (Kryo CPU + Adreno GPU + Hexagon DSP + Spectra ISP) enables efficient workload distribution across purpose-built processing units.

6. Edge AI Inference Frameworks

6.1 NVIDIA TensorRT

TensorRT is the gold standard for optimized inference on NVIDIA hardware. The TensorRT 10.x release introduces several robotics-critical features: dynamic shape support for variable-resolution camera inputs, INT8 calibration with minimal accuracy loss (typically <0.5% mAP degradation), and the TensorRT-LLM extension enabling on-device language model inference for robot instruction understanding. TensorRT achieves 2-5x inference speedup over native PyTorch on identical Jetson hardware through operator fusion, precision calibration, and memory-efficient kernel selection.

6.2 ONNX Runtime

ONNX Runtime provides a vendor-agnostic inference engine that supports deployment across Jetson (CUDA/TensorRT EP), Intel (OpenVINO EP), Qualcomm (QNN EP), and CPU-only platforms from a single model artifact. For robotics teams deploying heterogeneous fleets with mixed hardware, ONNX Runtime eliminates the need for platform-specific model optimization pipelines. The ONNX Runtime 1.18+ release includes a dedicated Robotics Execution Provider with optimized memory allocation patterns for streaming inference on sequential sensor data.

6.3 Intel OpenVINO

OpenVINO (Open Visual Inference and Neural Network Optimization) is Intel's inference toolkit, optimized for Intel CPUs, integrated GPUs, VPUs, and FPGAs. Its Model Optimizer converts models from PyTorch, TensorFlow, and ONNX formats into an Intermediate Representation (IR) with automatic layer fusion, constant folding, and precision optimization. OpenVINO's Neural Network Compression Framework (NNCF) provides post-training quantization and quantization-aware training with minimal accuracy degradation.

7. Fog Computing Architecture

7.1 Three-Tier Compute Architecture

Fog computing introduces an intermediate processing layer between on-robot edge devices and the centralized cloud, creating a three-tier architecture that optimally distributes workloads based on latency, compute intensity, and data locality requirements. In a typical robotic deployment, the fog layer consists of facility-local servers (often ruggedized rack-mount or edge micro-datacenter units) that provide 10-100x the compute capacity of individual robot edge modules while maintaining sub-5ms network latency within the facility.

7.2 Fog Node Hardware Specification

A production fog node for supporting 20-50 robots typically requires the following hardware profile:

• Compute: Dual NVIDIA A2 or L4 GPUs (or single A30) providing 40-140 TOPS aggregate inference capacity for fleet-level analytics, multi-camera tracking, and model serving
• CPU: AMD EPYC 7003 or Intel Xeon Scalable 4th Gen with 16-32 cores for ROS 2 fleet orchestration, database operations, and network processing
• Memory: 128-256GB DDR5 ECC RAM for maintaining fleet state, HD map caches, and concurrent model versions
• Storage: 2TB NVMe SSD (RAID-1) for model artifacts, map databases, and telemetry buffering; 20TB HDD for video recording retention
• Network: Dual 10GbE or 25GbE for robot fleet connectivity; dedicated management port for remote administration
• Form factor: Ruggedized rack-mount (IP52+) or edge micro-datacenter (Schneider Electric, Rittal) with integrated cooling for facility deployment

8. Edge-Cloud Hybrid Patterns

8.1 Split Inference Architecture

Split inference partitions a neural network between edge and cloud execution points, with early layers running on-device and deeper layers executed remotely. The split point is chosen to minimize the intermediate feature tensor size (the "bottleneck" layer), reducing bandwidth requirements by 10-100x compared to transmitting raw input data. This pattern is particularly effective for large vision transformers where the full model exceeds on-device memory but the early feature extraction fits comfortably.

In practice, split inference is most applicable to non-safety-critical perception tasks like scene understanding, activity recognition, and semantic mapping, where the 20-50ms additional latency from the cloud portion is acceptable. Safety-critical inference (obstacle detection, emergency stop) always remains fully on-device.

8.2 Edge Training with Federated Learning

Federated learning enables model improvement using data distributed across a fleet of robots without centralizing raw sensor data. Each robot performs local training updates (gradient computation) on its edge hardware, sending only model weight deltas to the cloud aggregation server. This approach provides three critical benefits for robotics: compliance with data sovereignty regulations (raw camera feeds never leave the facility), bandwidth efficiency (weight updates are 1000x smaller than training data), and continuous model improvement from fleet-wide operational experience.

9. Containerized Edge Deployment

9.1 Docker on Jetson: Production Patterns

Containerization has become the standard deployment methodology for edge robotics software stacks, providing reproducible builds, isolated dependencies, and streamlined OTA updates. NVIDIA provides official L4T (Linux for Tegra) base containers with pre-configured CUDA, cuDNN, and TensorRT libraries optimized for each Jetson module. Production deployments typically layer application containers on top of these base images, with Docker Compose orchestrating the multi-container robot software stack.

9.2 K3s: Lightweight Kubernetes for Robot Fleets

K3s, Rancher's lightweight Kubernetes distribution, has become the orchestration platform of choice for managing containerized workloads across robot fleets and fog nodes. At under 100MB binary size and 512MB RAM overhead, K3s runs comfortably on Jetson modules while providing the full Kubernetes API for deployment management, service discovery, secrets management, and rolling updates.

A typical K3s topology for robotic deployment uses the fog node as the K3s server (control plane) with each robot running a K3s agent. This enables centralized deployment management -- updating a perception model across 50 robots requires a single kubectl command to update the DaemonSet, with K3s handling rolling updates, health checks, and automatic rollback on failure.

10. OTA Model Updates & Lifecycle Management

10.1 Model Versioning & Registry

Over-the-air (OTA) model updates are essential for maintaining and improving robot perception performance without physical access to deployed units. A production OTA pipeline must handle three distinct artifact types: TensorRT engine files (hardware-specific, must be compiled per Jetson variant), ONNX model files (portable, compiled to TensorRT on-device), and configuration files (inference parameters, class mappings, confidence thresholds).

We recommend using an OCI-compliant container registry (Harbor, AWS ECR, or NVIDIA NGC Private Registry) as the model artifact store, with model versions embedded in container image tags. This approach leverages existing container pull infrastructure for model distribution and enables atomic updates -- the entire perception stack (model + inference code + configuration) is updated as a single versioned unit.

10.2 Safe Rollout Strategy

Model updates in safety-critical robotic systems require a staged rollout with automated validation gates:

1. Shadow mode (24-48h): New model runs alongside production model on 2-3 canary robots. Both models process identical inputs; only production model drives behavior. Outputs are compared for accuracy regression detection.
2. Canary deployment (48-72h): New model takes over primary inference on canary robots. Fleet monitoring tracks detection accuracy, latency P99, false positive rate, and safety stop frequency against baseline thresholds.
3. Progressive rollout: If canary metrics pass validation gates, K3s DaemonSet rolls the update to fleet segments (10% -> 25% -> 50% -> 100%) with automatic rollback if any segment degrades below threshold.
4. Full deployment + holdback: Previous model version retained on all robots as fallback. Automatic revert triggered if fleet-aggregate anomaly detection fires within 7-day observation window.

11. 5G + Edge for Mobile Robots

11.1 5G Capabilities for Robotics

5G networks -- particularly private 5G deployments in industrial facilities -- provide the connectivity foundation for a new class of edge-cloud hybrid robotic architectures. The three key 5G capabilities relevant to robotics are:

• Ultra-Reliable Low-Latency Communication (URLLC): Sub-5ms air interface latency with 99.999% reliability, enabling real-time fog-to-robot command distribution for coordinated multi-robot operations
• Enhanced Mobile Broadband (eMBB): 1-10 Gbps throughput supporting HD video offload from robots to fog nodes for fleet-level video analytics and remote monitoring
• Massive Machine-Type Communication (mMTC): Support for 1M+ devices per km2, enabling dense IoT sensor networks that augment robot perception with facility-wide environmental awareness

11.2 Multi-Access Edge Computing (MEC)

5G Multi-Access Edge Computing (MEC) places compute resources at the cellular base station or aggregation point, providing 1-5ms network latency to connected robots. For outdoor mobile robots (delivery, agriculture, security patrol), MEC eliminates the need for facility-local fog infrastructure by co-locating inference resources with the 5G radio network. Major APAC telecoms including Viettel (Vietnam), AIS (Thailand), and Singtel (Singapore) now offer MEC services with NVIDIA GPU instances at the network edge.

12. Edge Security & Power Optimization

12.1 Edge Security Architecture

Edge devices in robotic systems face a unique threat landscape: they operate in physically accessible environments (factory floors, public spaces), process safety-critical data, and connect to both local networks and cloud services. A defense-in-depth security architecture for edge robotics must address hardware root of trust, secure boot, encrypted model storage, network segmentation, and runtime integrity monitoring.

• Hardware root of trust: Jetson Orin's Secure Boot chain uses fused cryptographic keys to verify every firmware and software component before execution. Combined with Trusted Platform Module (TPM 2.0), this prevents unauthorized firmware modification even with physical device access.
• Model encryption: TensorRT engines contain proprietary IP and are vulnerable to extraction. Encrypt model files at rest using hardware-bound keys (Jetson Security Engine) and decrypt only into protected memory during inference.
• Network segmentation: Robot fleet traffic should be isolated on dedicated VLANs with firewall rules restricting communication to fog nodes and fleet management endpoints. Zero-trust networking (WireGuard VPN mesh) for robot-to-fog communication.
• Runtime attestation: Continuous integrity monitoring using dm-verity (read-only rootfs verification) and IMA (Integrity Measurement Architecture) to detect runtime tampering. Alert fleet management on any integrity violation.
• OTA security: All OTA updates signed with Ed25519 keys; update packages verified against organization-specific certificate chain before installation. Dual-partition (A/B) boot scheme ensures recovery from failed or compromised updates.

12.2 Power Consumption Optimization

For battery-powered robots, edge compute power consumption directly impacts mission duration and fleet economics. A warehouse AMR with a 48V/30Ah LiFePO4 battery pack (1.44 kWh) dedicating 25W to its Jetson Orin NX edge computer loses 17% of total battery capacity to compute alone. Optimizing power consumption extends mission time, reduces fleet size requirements, and lowers charging infrastructure costs.

Additional power optimization techniques for edge robotics include:

• DLA offloading: NVIDIA's Deep Learning Accelerator cores provide 2-3x better TOPS/watt than GPU execution for supported layers. Run primary detection models on DLA, reserving GPU for complex operations like transformer attention and custom CUDA kernels.
• Dynamic resolution scaling: Reduce camera input resolution during low-complexity scenes (empty corridors) and increase to full resolution in high-density areas (pick stations, intersections). A 50% resolution reduction yields approximately 4x inference speedup.
• Inference scheduling: Run perception models at reduced frequency (5 Hz instead of 30 Hz) when the robot is stationary or moving slowly. Scale inference frequency with velocity using a linear or sigmoid mapping.
• Peripheral power gating: Disable unused CSI camera interfaces, USB ports, and PCIe lanes when not in active use. Jetson's power rail management allows per-peripheral shutdown.

13. APAC Edge Infrastructure

13.1 Vietnam Edge Deployment Landscape

Vietnam's rapidly expanding industrial sector presents both opportunities and challenges for edge computing in robotics. The country's manufacturing-oriented FDI growth (particularly in electronics assembly around Bac Ninh, Thai Nguyen, and Hai Phong) is creating demand for automated quality inspection systems that require on-premises edge AI processing to meet production line cycle times of 2-5 seconds per unit.

• Network infrastructure: Vietnam's 5G rollout, led by Viettel and VNPT, covers major industrial zones in Hanoi, Ho Chi Minh City, and Da Nang. Private 5G spectrum allocation for industrial use is available under VNPT's enterprise 5G program, enabling facility-dedicated networks with guaranteed SLAs.
• Power reliability: Industrial parks in Tier 1 zones (VSIP, Amata, Long Hau) provide 99.9%+ power availability. Edge deployments in Tier 2/3 zones should include UPS (minimum 30-minute runtime) and graceful degradation firmware that transitions robots to safe-stop mode during power events.
• Environmental conditions: Ambient temperatures of 35-42C in non-air-conditioned factory areas require active cooling for fog servers and thermal-aware workload scheduling for Jetson modules. Conformal coating is recommended for robot edge hardware exposed to humidity above 80%.
• Local talent: Vietnamese universities (HUST, VNU, FPT University) produce strong embedded systems and AI engineering graduates. Seraphim Vietnam partners with these institutions for edge robotics training programs.

13.2 Regional Edge Infrastructure Comparison

13.3 Singapore as APAC Edge Hub

Singapore's position as the APAC technology hub makes it the ideal location for edge computing centers of excellence serving regional robotic deployments. Equinix's SG-series facilities and AWS Local Zones in Singapore provide sub-2ms latency to edge devices within the city-state and sub-20ms to major APAC industrial zones via submarine cable connectivity. For multi-country robotic fleet operators, Singapore serves as the centralized model training and fleet analytics hub with fog nodes deployed at each operational facility.

14. Implementation Roadmap

14.1 Phased Edge Deployment Strategy

Deploying edge computing infrastructure for robotic fleets requires a structured approach that balances technical validation with operational readiness. We recommend a four-phase implementation methodology:

1. Phase 1 -- Assessment & Prototyping (Weeks 1-6): Evaluate workload latency requirements, sensor data volumes, and connectivity constraints. Build proof-of-concept on Jetson development kits with representative models. Benchmark inference latency, power consumption, and thermal behavior under sustained load. Deliverable: hardware selection report and reference architecture.
2. Phase 2 -- Platform Engineering (Weeks 7-14): Build containerized software stack (Docker + K3s), establish CI/CD pipeline for model compilation and deployment, configure fog node infrastructure, and implement OTA update mechanism. Establish security baseline with secure boot, model encryption, and network segmentation. Deliverable: production-ready platform with 3 pilot robots.
3. Phase 3 -- Fleet Deployment (Weeks 15-22): Roll out edge compute modules across full robot fleet using K3s DaemonSets. Deploy fog nodes at each facility. Implement fleet monitoring dashboards (Grafana + Prometheus) with alerts for inference latency degradation, thermal throttling, and model accuracy drift. Deliverable: fully operational fleet with edge AI.
4. Phase 4 -- Optimization & Scale (Ongoing): Continuous model optimization via quantization refinement and architecture search. Implement federated learning for fleet-wide model improvement. Expand edge capabilities with new model types (language understanding, anomaly detection). Evaluate next-generation hardware (Jetson Thor, Hailo-15) for fleet refresh cycles. Deliverable: quarterly performance improvement reports.

14.2 Edge Computing TCO Model