Strands Robots: Deploy Policies to SO101 Hardware

Five separate tools currently fragment the workflow from dataset recording to hardware deployment, a gap the Strands Robots SDK closes by unifying them into one agent loop. Readers will learn how Strands Agents orchestrate this unified stack to record LeRobotDatasets in simulation that match hardware formats exactly, ensuring smooth policy transfer. The text details the internal mechanics of the agent loop, where GR00T and LerobotLocal serve policy inference behind a common interface while MolmoAct2 checkpoints run through the local path. Finally, the guide covers deploying these policies from MuJoCo-backed simulation directly to SO-101 hardware, using a Zenoh peer mesh to coordinate remote robots without rewriting core logic.

This approach builds on a legacy of mobile robot research, echoing the ambitions of the original STRANDS initiative which secured substantial funding from the European Union's Seventh Framework Programme to develop robots capable of learning from experience. By exposing these capabilities through a Python library, Strands Robots allows operators to control physical units with natural language, handling everything from servo calibration to real-time control loops within a single framework.

The Role of Strands Agents and LeRobot in Modern Robot Orchestration

Strands Robots SDK and LeRobot Integration Architecture

Strands Robots merges recording, training, simulation, deployment, and coordination into a single agent loop. This open-source SDK from AWS, licensed under Apache 2.0, exposes robot abstractions and the LeRobot stack as AgentTools. Historically, five separate tools handled these tasks: one to record new demonstrations, another to train, a third to test in simulation, custom code to deploy on hardware, and yet another to coordinate fleets. Those pieces worked in isolation without communicating. Strands Robots integrates natively with the LeRobot stack for hardware abstraction, offering a distinct advantage for users invested in the LeRobot system compared to generic agent frameworks.

Executing Sim-to-Real Transfer with LeRobotDataset Format

Sim-to-real transfer relies on a shared data structure across environments. Strands Robots achieves this by ensuring simulation tools record LeRobotDatasets in the exact on-disk format as physical devices. The command `Robot("so100")` defaults to a MuJoCo-backed simulation, requiring no hardware, while `mode="real"` switches to physical execution. This shared DatasetRecorder eliminates format conversion scripts that typically introduce latency or data loss during policy training.

Advanced policy inference within this unified loop requires an NVIDIA GPU with at least 16 GB of video memory, creating a hardware barrier for some developers. The agent loop remains constant across both domains, allowing operators to validate logic in a safe virtual environment before deploying to a physical SO-101. This architectural choice means a policy trained on synthetic data can immediately control real-world actuators without retraining. Builders gain the ability to iterate rapidly on logic without risking mechanical damage, though they must manage the computational cost of high-fidelity physics. The shared dataset format ensures that the transition from virtual testing to physical deployment is a configuration flag change rather than a data engineering project.

Infrastructure Risks in Transitioning from EU Grants to Self-Hosted GPU Clusters

Moving from the historic funded initiative to community development shifts financial liability for compute resources directly to the operator. The original STRANDS project absorbed research overhead, whereas current workflows require users to provision NVIDIA RTX GPU hardware locally. Simulation environments specifically require Isaac Sim 6.0 and Ubuntu 22.04+ as part of the system requirements, creating a high barrier for entry without institutional backing. The Apache 2.0 license eliminates software fees, yet the capital expenditure for high-end GPU hardware remains a significant operational risk for individual researchers. Developers can mitigate immediate procurement costs by using the default `mode="sim"` configuration, which runs entirely on CPU-backed MuJoCo instances for the default path. Transitioning to physical deployment or advanced policy inference reintroduces the requirement for dedicated graphics accelerators with substantial video memory. Infrastructure costs now dictate the scale of experimentation rather than grant availability. Teams must budget for hardware refreshes and power consumption that were previously covered by public funding mechanisms. This shift enables rapid iteration but removes the financial safety net that allowed long-horizon autonomy research to flourish without immediate ROI pressure.

Inside the Agent Loop and Sim-to-Real Data Flow

The Dual-Mode Robot Factory and LeRobotDataset Schema

Default behavior for `Robot("so100")` targets a MuJoCo-backed simulation, removing the immediate need for physical hardware during initial data collection. This specific setup guarantees the DatasetRecorder class outputs a LeRobotDataset with an identical parquet schema and per-camera MP4 layout regardless of whether the source is virtual or physical. Simulation tools capture these datasets in the exact on-disk format used by hardware, allowing the agent loop to process demonstration data without any format conversion logic.

Developers start this workflow by cloning the repository and executing the example script located at examples/lerobot/hub_to_hardware.py. Shared recording mechanisms remove the traditional friction where sim-to-real transfer fails because training and deployment environments use different schemas. Relying on a unified recorder means sensor noise from physical cameras does not appear in the default simulation path, which can cause policies to overfit to clean synthetic visuals. Operators must inject explicit noise or apply domain randomization during simulation to maintain robustness when switching the agent to `mode="real"`. This architectural constraint forces an early choice between data fidelity and development speed.

Orchestrating GR00T Inference Containers and Local Policies

Execution starts when the agent calls `gr00t_inference` with `action="lifecycle"` to pull the container and launch the service on a assigned port. This command automates deployment of isolated Docker services hosting the policy server, keeping the environment portable across different host configurations. The system depends on a ZMQ inference client to maintain low-latency data transfer between the agent loop and the remote policy process. Operators must provision an NVIDIA GPU with at least 16 GB of video memory to sustain the tensor operations required by the GR00T model.

Developers seeking reduced infrastructure overhead may choose `LerobotLocalPolicy`, which performs inference directly inside the Python process. This approach completely removes the need for container orchestration or ZeroMQ networking. Loading these models requires setting the environment variable `STRANDS_TRUST_REMOTE_CODE=1` to enable the `trust_remote_code=True` flag.

Isolation safety competes with latency predictability in this design. Containerization prevents library conflicts, yet the network hop introduces jitter that in-process calls avoid. Teams prioritizing strict security boundaries should adopt the containerized GR00T path, whereas latency-critical loops benefit from the direct memory access of local policies.

Pre-Deployment Checklist: HF Tokens, Mock Policies, and Real Modes

Moving from simulation to physical deployment requires verifying Hugging Face credentials before attempting a dataset push. A Hugging Face token is optional when using the Mock policy for local testing, yet it becomes mandatory to upload recorded demonstrations to the Hub. The Mock policy generates structurally valid but functionally useless actions, serving only to validate the recording pipeline rather than train capable agents.

Push errors frequently occur due to missing write permissions on the target repository rather than format mismatches. The LeRobotDataset schema remains constant across environments, so authentication failures are the primary blocker during the upload phase. Switching to physical execution involves changing exactly one keyword argument: `mode="real"`. This single parameter shift routes the Robot factory from the MuJoCo simulation backend to the hardware drivers without altering the agent logic or data structures. Trusting that a policy performing well in a perfect physics engine will handle real-world friction creates operational tension. Simulation removes mechanical latency, whereas hardware introduces unmodeled dynamics that can destabilize controllers tuned solely on synthetic data.

Deploying Policies from Simulation to SO-101 Hardware

Implementation: GR00T Inference Containers and LerobotLocalPolicy Modes

Infrastructure footprint depends entirely on the choice between containerized GR00T services and in-process local policies. Running a policy in simulation requires the agent to attach `gr00t_inference` for managing the inference container lifecycle. This isolation strategy demands an NVIDIA GPU with at least 16 GB of video memory and Docker installed. A ZMQ inference client handles low-latency data transfer between the agent loop and the remote policy server living inside that isolated service.

Developers might prefer `LerobotLocalPolicy` for execution without container overhead or ZeroMQ networking. Models load directly into the host Python process, supporting architectures like ACT and Diffusion Policy. Loading models with `trust_remote_code=True` forces the environment variable `STRANDS_TRUST_REMOTE_CODE=1`. Portability comes with memory overhead while local execution reduces latency but increases host dependency complexity.

1. Export `STRANDS_TRUST_REMOTE_CODE=1` if using local policies.
2. Verify Docker daemon status for GR00T container deployment.
3. Choose `gr00t_inference` for isolated, versioned policy serving.
4. Select `LerobotLocalPolicy` for minimal-latency, single-host scenarios.

The `gr00t_inference` tool's `lifecycle="full"` action manages container lifecycles, while `LerobotLocalPolicy` executes within the host process.

Switching to Real Mode for SO-101 Hardware Deployment

Physical operation starts by changing the `Robot` factory argument to `mode="real"` to engage LeRobot drivers instead of the default MuJoCo backend. One parameter swap redirects the agent loop from virtual physics to actual servo control ports, such as `/dev/ttyACM0`, while maintaining the exact same policy interface.

1. Verify that calibration files exist in `~/.cache/huggingface/lerobot/calibration/` for the specific SO-101 follower and leader pair.
2. Update the initialization code to explicitly set `mode="real"` and define camera paths like `/dev/video0`.
3. Execute the agent loop, which now streams actions to hardware rather than a simulated environment.

The underlying LeRobotDataset format remains unchanged during this transition, ensuring that policies trained on MuJoCo Simulation execute correctly on Physical SO-101 Hardware without schema conversion. The `Robot("so100")` command defaults to a simulation instance to prevent hardware risk, while `mode="real"` returns a hardware-backed robot driven by LeRobot. For hardware recording and calibration, LeRobot's own CLIs (`lerobot-record`, `lerobot-calibrate`) handle the bring-up; the agent picks up from there.

Mitigating Prompt Injection Risks in Physical Robot Control

Untrusted natural language inputs can trigger destructive physical actions when agents control real hardware without validation layers. Prompt injection poses a genuine threat when supplying untrusted data to agents controlling physical robots.

Configuring `STRANDS_MESH_AUTH_MODE=mtls` enforces mutual authentication since the `STRANDS_MESH_LOCAL_DEV=1` setting explicitly disables security checks for local testing. This configuration prevents unauthorized nodes from injecting commands into the agent mesh during physical hardware interaction.

1. Implement human-in-the-loop interrupts to halt autonomous execution of critical movement commands.
2. Validate all text prompts against a allowlist before passing them to the policy engine.
3. Isolate the robot control network from public internet access to reduce attack surface.

The peer mesh based on Zenoh allows the agent to coordinate remote robots, requiring careful configuration of authentication modes between local development and production networks. Builders should treat natural language as an untrusted boundary and enforce strict authentication settings before enabling real-mode actuation.

Strategic Advantages of Zenoh Mesh for Robot Fleet Coordination

Zenoh Peer-to-Peeer Mesh vs Brokered IP Architectures

Strands Robots removes manual IP management by using a native Zenoh peer-to-peer mesh for fleet discovery and command broadcasting. Traditional brokered architectures demand central server maintenance, whereas this approach lets agents locate peers and execute emergency stops dynamically without static configuration files. Data flows directly between nodes using content-based routing rather than fixed addresses because the message broker is gone.

Deploy this mesh topology when coordinating multiple robots where network conditions fluctuate or IP addresses change frequently. The system supports structured commands and broadcasts natively, enabling parallel execution of tasks like "go to home pose" across the entire fleet. For cloud-integrated scenarios, the `[mesh-iot]` extra routes traffic through AWS IoT Core using MQTT5 with mTLS, bridging local discovery with secure wide-area networks.

Operational tension exists between development convenience and production security. Setting `STRANDS_MESH_LOCAL_DEV=1` disables authentication to simplify local testing, yet this configuration creates vulnerability to prompt injection in untrusted networks. Production deployments must enforce `STRANDS_MESH_AUTH_MODE=mtls` to validate node identity before accepting control commands. Human-in-the-loop interrupts remain mandatory for critical actions to prevent autonomous execution of potentially destructive physical moves. This architecture shifts the burden from network engineering to policy enforcement, allowing engineers to focus on robot behavior rather than connectivity plumbing.

Executing Parallel Broadcast Commands via robot_mesh Tool

The `robot_mesh` tool executes parallel commands like "go to home pose" across all discovered peers without managing individual IP addresses. This capability addresses the coordination friction found when scaling from single-unit simulation to multi-robot fleets. Operators trigger these broadcasts using the Robot class, which abstracts the underlying peer-to-peer discovery logic.

Physical safety imposes a critical operational constraint. By default, physically actuating mesh actions such as `broadcast`, `emergency_stop`, or `stop` require human approval via an interrupt. This mechanism prevents autonomous agents from executing destructive movements if a policy hallucinates due to prompt injection. Developers control this behavior through the `STRANDS_MESH_HITL_ACTIONS` environment variable.

Deployment velocity clashes with operational safety here. Simulation allows rapid iteration of fleet logic, but skipping human-in-the-loop checks on hardware introduces immediate physical risk. The mesh architecture enables rapid command propagation, yet the system deliberately inserts latency for critical actions to ensure human oversight. This design choice prioritizes fleet integrity over raw execution speed during the transition from virtual testing to real-world deployment.

Strands Device Connect vs Native Zenoh Mesh for Cloud Fleets

Production fleets requiring AWS IoT Core routing must install the `[mesh-iot]` extra to tunnel Zenoh traffic over MQTT5 with mTLS. This configuration shifts the discovery mechanism from local peer-to-peer broadcasting to a managed cloud topology suitable for wide-area networks. The cost is latency dependent on cloud round-trips, contrasting with the sub-millisecond response of local mesh networks. Native mesh excels in contained facilities, while cloud-connected operations demand the certificate management that Device Connect provides.

Developed with Arm, Device Connect acts as the primary coordination layer for production environments, handling safety-critical discovery before falling back to the built-in Zenoh mesh if cloud services become unavailable. This hybrid approach ensures continuous operation even during intermittent connectivity, a scenario where pure cloud-dependant architectures fail. The drawback remains the added complexity of maintaining mTLS certificates alongside local network credentials.

Enabling `[mesh-iot]` fundamentally alters the failure domain from network partitions to cloud service availability. The STRANDS_MESH_AUTH_MODE=mtls setting becomes mandatory rather than optional when bridging these environments to prevent unauthorized node injection. Generic agent frameworks often lack native hardware abstraction, but this integration uses specific AgentTools to bridge reasoning and actuation securely. The architectural decision ultimately rests on whether the fleet operates within a single broadcast domain or spans multiple geographic locations requiring centralized oversight.

References

• OpenCode 2026: Setup Guide for the Top Open-Source AI
• AI Agent Orchestration for Developers: The Complete 2026 Guide
• Deploy OpenHands on GPU Cloud: Self-Host the Open-Source AI