Strands Robots: Deploy Policies to SO101 Hardware

Five separate tools currently fragment the workflow from dataset recording to hardware deployment, a gap the Strands Robots SDK closes by unifying them into one agent loop. Simulation records LeRobotDatasets in the on-disk format hardware already uses, so a policy validated in MuJoCo reaches an SO-101 arm without a data engineering step in between. GR00T and LerobotLocal serve policy inference behind a common interface, with MolmoAct2 checkpoints running through the local path, and a Zenoh peer mesh coordinates remote robots without rewriting the agent logic.

This approach builds on a long legacy of mobile robot research aimed at machines that learn from experience. By exposing these capabilities through a Python library, Strands Robots allows operators to control physical units with natural language, handling everything from servo calibration to real-time control loops within a single framework.

The Role of Strands Agents and LeRobot in Modern Robot Orchestration

Strands Robots SDK and LeRobot Integration Architecture

Strands Robots merges recording, training, simulation, deployment, and coordination into a single agent loop. This open-source SDK from AWS, licensed under Apache 2.0, exposes robot abstractions and the LeRobot stack as AgentTools. Historically, five separate tools handled these tasks: one to record new demonstrations, another to train, a third to test in simulation, custom code to deploy on hardware, and yet another to coordinate fleets. Those pieces worked in isolation without communicating. Strands Robots integrates natively with the LeRobot stack for hardware abstraction, offering a distinct advantage for users invested in the LeRobot system compared to generic agent frameworks.

Executing Sim-to-Real Transfer with LeRobotDataset Format

Sim-to-real transfer relies on a shared data structure across environments. Strands Robots achieves this by ensuring simulation tools record LeRobotDatasets in the exact on-disk format as physical devices. The command Robot("so100") defaults to a MuJoCo-backed simulation, requiring no hardware, while mode="real" switches to physical execution. This shared DatasetRecorder eliminates format conversion scripts that typically introduce latency or data loss during policy training.

Advanced policy inference within this unified loop requires a dedicated NVIDIA GPU, and advanced simulation setups add Isaac Sim 6.0 and Ubuntu 22.04+ to the requirement list, while the default mode="sim" path runs on CPU-backed MuJoCo instances. The agent loop remains constant across both domains, allowing operators to validate logic in a safe virtual environment before deploying to a physical SO-101. This architectural choice means a policy trained on synthetic data can immediately control real-world actuators without retraining. Builders gain the ability to iterate rapidly on logic without risking mechanical damage, though they must manage the computational cost of high-fidelity physics. The shared dataset format ensures that the transition from virtual testing to physical deployment is a configuration flag change rather than a data engineering project.

Inside the Agent Loop and Sim-to-Real Data Flow

The Dual-Mode Robot Factory and LeRobotDataset Schema

Default behavior for Robot("so100") targets a MuJoCo-backed simulation, removing the immediate need for physical hardware during initial data collection. This specific setup guarantees the DatasetRecorder class outputs a LeRobotDataset with an identical parquet schema and per-camera MP4 layout regardless of whether the source is virtual or physical.

Developers start this workflow by cloning the repository and executing the example script located at examples/lerobot/hub_to_hardware.py. Shared recording mechanisms remove the traditional friction where sim-to-real transfer fails because training and deployment environments use different schemas. Relying on a unified recorder means sensor noise from physical cameras does not appear in the default simulation path, which can cause policies to overfit to clean synthetic visuals. Operators must inject explicit noise or apply domain randomization during simulation to maintain robustness when switching the agent to mode="real". This architectural constraint forces an early choice between data fidelity and development speed.

Orchestrating GR00T Inference Containers and Local Policies

Execution starts when the agent calls gr00t_inference with action="lifecycle" to pull the container and launch the service on an assigned port. This command automates deployment of isolated Docker services hosting the policy server, keeping the environment portable across different host configurations. The system depends on a ZMQ inference client to maintain low-latency data transfer between the agent loop and the remote policy process. Operators must provision an NVIDIA GPU with at least 16 GB of video memory to sustain the tensor operations required by the GR00T model.

Developers seeking reduced infrastructure overhead may choose LerobotLocalPolicy, which performs inference directly inside the Python process. This approach completely removes the need for container orchestration or ZeroMQ networking. Loading these models requires setting the environment variable STRANDS_TRUST_REMOTE_CODE=1 to enable the trust_remote_code=True flag.

Isolation safety competes with latency predictability in this design. Containerization prevents library conflicts, yet the network hop introduces jitter that in-process calls avoid. Teams prioritizing strict security boundaries should adopt the containerized GR00T path, whereas latency-critical loops benefit from the direct memory access of local policies.

HF Tokens, Mock Policies, and the Switch to Real Mode

Moving from simulation to physical deployment requires verifying Hugging Face credentials before attempting a dataset push. A Hugging Face token is optional when using the Mock policy for local testing, yet it becomes mandatory to upload recorded demonstrations to the Hub. The Mock policy generates structurally valid but functionally useless actions, serving only to validate the recording pipeline rather than train capable agents.

Push errors frequently occur due to missing write permissions on the target repository rather than format mismatches. The LeRobotDataset schema remains constant across environments, so authentication failures are the primary blocker during the upload phase. Trusting that a policy performing well in a perfect physics engine will handle real-world friction creates operational tension. Simulation removes mechanical latency, whereas hardware introduces unmodeled dynamics that can destabilize controllers tuned solely on synthetic data.

Deploying Policies from Simulation to SO-101 Hardware

The runtime chosen in simulation carries straight through to hardware, so this decision belongs before the first real-mode run. Local policies load models into the host Python process and support architectures like ACT and Diffusion Policy.

Switching to Real Mode for SO-101 Hardware Deployment

Physical operation starts by changing the Robot factory argument to mode="real" to engage LeRobot drivers instead of the default MuJoCo backend. One parameter swap redirects the agent loop from virtual physics to actual servo control ports, such as /dev/ttyACM0, while maintaining the exact same policy interface.

1. Verify that calibration files exist in ~/.cache/huggingface/lerobot/calibration/ for the specific SO-101 follower and leader pair.
2. Update the initialization code to explicitly set mode="real" and define camera paths like /dev/video0.
3. Execute the agent loop, which now streams actions to hardware rather than a simulated environment.

For hardware recording and calibration, LeRobot's own CLIs (lerobot-record, lerobot-calibrate) handle the bring-up; the agent picks up from there.

Mitigating Prompt Injection Risks in Physical Robot Control

Untrusted natural language inputs can trigger destructive physical actions when agents control real hardware without validation layers. Prompt injection poses a genuine threat when supplying untrusted data to agents controlling physical robots.

Configuring STRANDS_MESH_AUTH_MODE=mtls enforces mutual authentication, whereas the STRANDS_MESH_LOCAL_DEV=1 setting explicitly disables security checks for local testing. This configuration prevents unauthorized nodes from injecting commands into the agent mesh during physical hardware interaction.

1. Implement human-in-the-loop interrupts to halt autonomous execution of critical movement commands.
2. Validate all text prompts against an allowlist before passing them to the policy engine.
3. Isolate the robot control network from public internet access to reduce attack surface.

The peer mesh based on Zenoh allows the agent to coordinate remote robots, requiring careful configuration of authentication modes between local development and production networks. Builders should treat natural language as an untrusted boundary and enforce strict authentication settings before enabling real-mode actuation.

Strategic Advantages of Zenoh Mesh for Robot Fleet Coordination

Zenoh Peer-to-Peer Mesh vs Brokered IP Architectures

Strands Robots removes manual IP management by using a native Zenoh peer-to-peer mesh for fleet discovery and command broadcasting. Traditional brokered architectures demand central server maintenance, whereas this approach lets agents locate peers and execute emergency stops dynamically without static configuration files. Data flows directly between nodes using content-based routing rather than fixed addresses because the message broker is gone.

Deploy this mesh topology when coordinating multiple robots where network conditions fluctuate or IP addresses change frequently. The system supports structured commands and broadcasts natively, enabling parallel execution of tasks like "go to home pose" across the entire fleet. For cloud-integrated scenarios, the [mesh-iot] extra routes traffic through AWS IoT Core using MQTT5 with mTLS, bridging local discovery with secure wide-area networks.

This architecture shifts the burden from network engineering to policy enforcement, allowing engineers to focus on robot behavior rather than connectivity plumbing.

Executing Parallel Broadcast Commands via robot_mesh Tool

The robot_mesh tool executes parallel commands like "go to home pose" across all discovered peers without managing individual IP addresses. This capability addresses the coordination friction found when scaling from single-unit simulation to multi-robot fleets. Operators trigger these broadcasts using the Robot class, which abstracts the underlying peer-to-peer discovery logic.

Physical safety imposes a critical operational constraint. By default, physically actuating mesh actions such as broadcast, emergency_stop, or stop require human approval via an interrupt. This mechanism prevents autonomous agents from executing destructive movements if a policy hallucinates due to prompt injection. Developers control this behavior through the STRANDS_MESH_HITL_ACTIONS environment variable.

Deployment velocity clashes with operational safety here. Simulation allows rapid iteration of fleet logic, but skipping human-in-the-loop checks on hardware introduces immediate physical risk. The mesh architecture enables rapid command propagation, yet the system deliberately inserts latency for critical actions to ensure human oversight. This design choice prioritizes fleet integrity over raw execution speed during the transition from virtual testing to real-world deployment.

Strands Device Connect vs Native Zenoh Mesh for Cloud Fleets

Production fleets requiring AWS IoT Core routing must install the [mesh-iot] extra to tunnel Zenoh traffic over MQTT5 with mTLS. This configuration shifts the discovery mechanism from local peer-to-peer broadcasting to a managed cloud topology suitable for wide-area networks. The cost is latency dependent on cloud round-trips, contrasting with the sub-millisecond response of local mesh networks. Native mesh excels in contained facilities, while cloud-connected operations demand the certificate management that Device Connect provides.

Developed with Arm, Device Connect acts as the primary coordination layer for production environments, handling safety-critical discovery before falling back to the built-in Zenoh mesh if cloud services become unavailable. This hybrid approach ensures continuous operation even during intermittent connectivity, a scenario where pure cloud-dependant architectures fail. The drawback remains the added complexity of maintaining mTLS certificates alongside local network credentials.

Enabling [mesh-iot] fundamentally alters the failure domain from network partitions to cloud service availability, and STRANDS_MESH_AUTH_MODE=mtls becomes mandatory rather than optional at that bridge. The architectural decision ultimately rests on whether the fleet operates within a single broadcast domain or spans multiple geographic locations requiring centralized oversight.

References

• OpenCode 2026: Setup Guide for the Top Open-Source AI
• AI Agent Orchestration for Developers: The Complete 2026 Guide
• Deploy OpenHands on GPU Cloud: Self-Host the Open-Source AI