Docs / Guides
Robots: from simulation to real hardware
Training robots in simulation with EsaSim, what makes a policy work on the real robot, and where EsaSim is going: robot descriptions, robot categories and bridge workers that run policies on real devices.
Robots learn in simulation because it is fast and nothing breaks: thousands of copies of a robot try at once on a GPU, fall, and try again. The aim is a policy, the neural network that drives the robot, that then does the same on the real worker. This page describes what EsaSim does for that today, and the parts that are coming.
Available now: training in MuJoCo and MuJoCo Warp, the Humanoid block (Unitree G1, Unitree H1, Apptronik Apollo) and its project, the 3D view and telemetry of robots, runs spread over several workers. Planned: robot descriptions for any robot, robot categories, policy export and bridge workers that run policies on real hardware. Sections about planned features say so.
Training a robot today
The Humanoid block trains a humanoid robot in MuJoCo, with thousands of robots at once on an NVIDIA GPU through MuJoCo Warp, or a few hundred on the CPU to try it. Its Robot setting says which:
| Robot | Size | Model |
|---|---|---|
| Unitree G1 (the default) | 1.3 m, 35 kg, 29 joints | MuJoCo Menagerie (BSD-3) and MuJoCo Playground; 3D model by KenzoDkohno22T (CC BY 4.0) |
| Unitree H1 | 1.8 m, 51 kg, 19 joints | MuJoCo Menagerie, Unitree Robotics (BSD-3) |
| Apptronik Apollo | 1.7 m, 81 kg, 32 joints | MuJoCo Menagerie, Apptronik (Apache 2.0) |
Each robot has its own policies, and the file names say whose they are: humanoid_walk_… and humanoid_standup_… are the G1's, humanoid_h1_walk_… and humanoid_apollo_standup_… the others'. They can all sit in the same dataset: a run uses only the policies of the robot it is set to (Continue, Walk) and leaves the others as they are, so switching robot loses nothing. The rewards and settings were tuned on the G1; the H1 and the Apollo learn the same task scaled to their height and weight, and may need longer, or some tuning, to walk as well.
The project, Humanoid training, has five flows (each with the Robot setting on its Humanoid block):
| Flow | What it learns or does |
|---|---|
| 1 · Learn to stand up | Getting up from wherever a fall left it, with a help that is taken away as it succeeds. |
| 2 · Learn to walk | Walking straight on a flat floor at the speed asked, pushed now and then. |
| 3 · Learn the circuit | Rough ground, slopes, moving tiles, smooth ice and stairs, reading the ground ahead from the depth camera in its head. |
| 4 · Test the circuit | One robot walks the circuit live in 3D with the newest policies, steered from the view and pushed by hand (click Push, then the part of the robot to push), and gets up on its own after a fall. |
| 5 · Playground | You drive the robot by hand, with the drive pad in the view or the keyboard, over sand, a pool of water that drags at its feet, gravel, stepping stones, curbs, ice and steps. |
Things that apply to any long training:
- Continue or start over. The block's Training setting. Continue carries on from the chosen robot's policy with the most experience: the ones wired in from a dataset, and the checkpoint a run left on the worker when it was stopped before writing its policy. The optimizer's state comes with it, so learning picks up where it was. Start over learns from nothing.
- The time limit. A run stops at its Stop after time. The block reads the time left with
ctx.time_left()and ends its training three minutes before, so the policy is always written. Set Stop after a little longer than the block's Minutes; the run panel does this for you. - Watching it learn. An episode is replayed in the 3D view every few seconds, charts follow the scores, and a telemetry window shows every sensor and joint.
What makes a policy work on the real robot
A policy that is perfect in simulation can stumble on the real robot. The gap closes from two sides.
Train for the real robot
- Read only what the robot can sense. The policy may read the IMU (turn rates, which way is down), joint angles and speeds, foot contacts, cameras, and what it did last. Values only the simulator knows, such as the body's exact speed over the ground or the friction under its feet, may help training, but only the critic, the part of the training that judges actions, should read them. The critic is thrown away after training.
- Vary the world. Every robot of a training gets a slightly different friction, mass, motor stiffness and sensor noise, and is pushed now and then. A policy that copes with all of them copes with the real one.
- Add the delays. Real sensors and motors answer a few milliseconds late. A policy trained without delays oscillates on hardware.
The Humanoid block adds sensor noise, pushes and odd starting poses. Its policy still reads the body's speed from the simulation; a version meant for a real robot will work it out from the IMU and the joints instead.
Make the simulation match the robot
Measure the real robot: the friction in its joints, how fast its motors follow a target, its delays, the mass of what it carries. Put those numbers back into the simulation. This is system identification, and it is the other half of making the real robot behave as it did in training. Planned: measuring routines run by the bridge worker (below), with the results going back into the robot's description.
Robot descriptions Planned
To train any robot, EsaSim needs one file that describes it. The same file is used in simulation and on the real hardware, so a robot needs no glue code per project.
| Part | What it says |
|---|---|
| Body | Links with their shapes, masses and inertias; the shapes that touch the ground or each other |
| Joints | Type, axis, range, friction and damping |
| Motors | Which joint each drives, its torque and speed limits, its control (position, speed or torque) and gains |
| Sensors | IMUs, joint encoders, force and contact sensors, cameras and depth cameras, lidars, where each is mounted, its rate and noise |
| Hardware map | For each motor and sensor, how the real robot reaches it: a motor ID on a bus, a port, a topic; units, direction and zero offset |
| Limits | The joint, speed and torque limits the bridge enforces on the real robot, whatever the policy asks |
It starts from the formats robot makers already ship: MJCF (MuJoCo) and URDF. EsaSim adds the hardware map and the limits. You upload a robot's description once; every training and every bridge then uses it.
Robot categories Planned
Robots of the same kind learn the same skills from the same kind of inputs. A category is a template: what the policy reads, what it controls, the commands it follows, the rewards and the scores. Pick a category and a description, and training works for a robot EsaSim has never seen.
| Category | Typical skills | The policy controls |
|---|---|---|
| Legged (humanoids, quadrupeds) | Walk at a commanded speed, stairs and rough ground, getting up | Joint position targets |
| Arms and manipulators | Reach, grasp, place, insert | Joint or end-effector targets, the gripper |
| Wheeled and tracked | Follow a path, avoid obstacles, dock | Wheel speeds, steering |
| Flying | Hover, follow a trajectory, land | Rotor thrusts or attitude and thrust |
From a trained policy to the real robot Planned
- Export. The policy is exported with its settings: the order of the joints, the standing pose, the motor gains, the scale of its actions and exactly what it reads, in the order it reads it. Nothing is guessed on the robot.
- Check in the maker's simulator. Before the real robot, the exported policy runs in the robot maker's own simulator through the same interface the real robot uses (for a Unitree robot, its SDK over its network protocol). Mistakes in joint order or units show up here, not on hardware.
- Run it on a bridge worker. A local worker connected to the robot runs the policy against it, and EsaSim shows what the robot does in the same views as the simulation: the same 3D view, the same telemetry window, side by side with the simulation of the same commands.
Bridge workers Planned
A bridge worker is one of your local workers that you mark as connected to a real device. When you run a flow, you choose which bridge, and so which device, it uses. Real devices usually take part in a flow's simulation and testing, not its training: train on the digital twin, then plug in the real one and compare.
- Where it is. A bridge is a local worker, in the same network as the device or connected to it directly. Bridges are never cloud workers.
- The control loop runs on the bridge. A walking policy needs a command every 20 ms and a motor loop a few thousand a second: too fast for a round trip to EsaSim. The bridge block loads the exported policy and runs the loop on the worker, or loads it onto the device. EsaSim sends the policy and the settings, and receives telemetry and logs.
- Safety the bridge enforces. The limits of the robot's description, a watchdog that puts the motors in a safe state if the loop or the network stalls, and an emergency stop in the view that always works. A block cannot turn them off.
- What it finds. The bridge reports the devices it can reach (robots on its network, USB and serial ports, boards it recognises) the way a worker reports its GPUs today.
- Devices of every size. From robots with their own computer and SDK to microcontrollers. For boards such as ESP32, STM32 or Arduino, the bridge has the compiler and the flasher (PlatformIO covers most boards), so a block can build and load firmware, or talk to firmware already running on the board.
Start every real robot hanging in a harness, in its damping or debug mode, with the emergency stop at hand. Keep skills such as getting up in simulation until they are reliable there: a fall can break a real robot.
Learn more
Runs and jobs (spreading a run over several workers), Build your own blocks, 3D scenes and rendering.