36Simulation vs reality: why sim is used, and how it lies

Thirty-five modules in, you have driven an arm, a rover, a four-legged dog and a drone, and not one of them exists. Every robot in this course has been a simulation: a computer working out, step by step, what a described world would do. The description is a long list of numbers, the shape and mass of every part, how grippy every surface is, how strong every motor is, and the rules are physics. The arm you are about to see is stepped forward two hundred times a second, and in each step the computer asks what gravity, the motors and every touching surface do in the next five thousandths of a second, and moves everything that much.

That has been the quiet deal of this course. Simulation is how robots are planned, tested and increasingly taught, and it is also a liar, in two particular ways. The box below runs the arm from the module on grippers twice, side by side, on one plan, and the two worlds disagree about one small block.

Why anyone simulates

Think about what finding the module on grippers' numbers would have cost on a real bench. That module's claim, that a slippery block is squeezed out at any grip force, came from running the same pick-and-place over and over with the friction and the grip force set to different values: dozens of runs, each about ten seconds, each from exactly the same starting pose. On a bench that is a day of resetting a block by hand to the millimeter and swapping blocks of different plastics. In simulation it is a script left running for a minute, because nothing says the steps have to keep pace with a clock: with no screen to draw, the same computer takes thousands of steps a second instead of two hundred, and every run starts from the identical state, which reality never offers twice.

So simulation is used for everything that is slow, costly or dangerous to try for real. A plan can be tried before the robot is built. A dog can be dropped off a table a thousand times to see how it lands, and nothing breaks. A controller like the one from the module on PID can have its three pushes tuned by trying every setting overnight.

How a simulation lies

The catch is the list of numbers, which is called the model. Someone typed every number in it, and a few of them are guesses. In the module on grippers the words said the block and the jaws are hard plastic, so the grippiness between them is about 0.2. That number is a line in a file. If the block on the real table is a different plastic, or dusty, or the jaws are worn smooth, the real grippiness is something else, and the simulation has no way of knowing. It will go on predicting, with complete confidence, what a 0.2 block would do.

That is the first way a simulation lies: a number it was given is wrong. The second is worse. Some things are not in the list at all. In this simulation the jaws are rigid and the block is a perfect cube that never dents. A real jaw flexes a little under 37 newtons, a real rubber pad flattens against the block and grips several times better than plastic, a real motor warms up over an hour and gets weaker, and every real sensor reports a little wrongly. A thing that is not described cannot happen in the simulation, however much it happens on the bench.

The difference between what the simulation predicts and what the real robot does is the reality gap. Every robot that has ever been planned in simulation has had to cross it, and the plan below is about to fall into it.

The box

Two copies of the arm, the block and the bin from the module on grippers, side by side on one clock. Both run the same plan: the same fixed script of reach, lower, close, lift, carry and release, with the jaw motor at its full 2.94 newton meters.

On the left, labeled Reality, is the block that turned up on the real table: a slippery plastic, like the coating on a frying pan, with a grippiness of 0.08, weighing a hundred grams. Nothing on that side can be changed, because nobody gets to turn a dial on reality. On the right, labeled The simulation, is the block as the model has it: hard plastic, grippiness 0.2, fifty grams.

Between them, under The model's error, are two sliders, Grippiness and Mass, one for each of the model's numbers. Beside its name each says how the model's number stands against the real one: as the box opens, too grippy and too light. A tick under each slider marks where the two are equal.

Be clear about one thing: the left side is a simulation too. A real robot cannot be put on a web page, so the left arm stands in for reality by being a simulation that was given reality's numbers. That is not a cheat; the last try depends on it.

The box opens paused, and one press of Play runs both arms together. Under each arm a line says what the arm is doing and what became of its block, and a bold line gives the block's two numbers. Moving a slider starts both arms over, paused.

Reality

Loading the simulation. The robot runs in your browser, so the first one takes a few seconds to arrive.

The model's error

Grippiness: too grippyexacttoo grippyMass: too lighttoo lighttoo heavy

Moving a slider starts both sides over.

The simulation

Loading the simulation. The robot runs in your browser, so the first one takes a few seconds to arrive.

ResetPauseSlowLabels

This is easier on a larger screen, but it works here too.

One plan, two worlds

Nothing in the plan is wrong. It was written for the block in the model, and for that block it works every time, as the right side shows. The left arm did every motion identically, to the hundredth of a second, because it is the same script on the same clock. What differed was the world the script met.

The module on grippers explained the failure it met. The swinging jaw stalls about 13 degrees short of parallel, so the block sits in a shallow V open toward the fingertips, and a slice of the squeeze leaks along the faces, out of the jaws. Friction holds the block against that leak, and friction is also a slice of the squeeze. Squeeze harder and both grow together, so the squeeze cancels out and the grippiness alone decides: above about 0.10 the block stays, below it the block walks out however hard the jaws press. The simulation's block, at 0.2, is safely above the line. Reality's block, at 0.08, is below it, and the plan had no way to know.

Which lie mattered

The model is wrong about two numbers, the block's grippiness and its mass, both lies of the first kind. A plan that fails on the real robot does not say which wrong number failed it. Finding out is most of the job, and it is done the way a careful person does anything: change one thing at a time. The simulation is the tool for that, because on a copy that costs nothing you can put one number right and run the exact same situation again.

The weight was innocent. At full squeeze the jaws press the block with about 37 newtons, and even on reality's block, at a grippiness of 0.08, two faces give about 6 newtons of hold, six times the weight of the hundred-gram block, which never comes near that limit. And inside the V the weight barely enters the contest: the leak and the friction both scale with the squeeze, and the block's half newton or one newton is small beside either. The simulation was wrong about the mass by a factor of two, and putting that right changed nothing. That leaves the grippiness, which the model has wrong by a bit more than a factor of two.

That is the general shape of the reality gap. A simulation is wrong about dozens of numbers at once, most of them harmlessly, and one or two of them fatally, and nothing in the simulation tells you which is which. Only the real robot does, by failing, and then the simulation, by being corrected one number at a time until it fails the same way.

Closing the gap

Given reality's number, the simulation predicts reality, here to the instant, because both sides are the same program. That is the whole method, and it has a name: sim-to-real transfer is taking a plan that works in simulation and getting it to work on the real robot, and most of that work is measuring. Weigh the block. Put it on a board of the jaws' plastic and tilt the board slowly: tilting turns part of the block's weight into a pull along the board and leaves the rest pressing into it, and the block goes at the slope where the pull along equals the grippiness times the press. So the slope it slides at, the board's rise for each centimeter along, is the grippiness. A block of 0.2 goes at a rise of 2 in 10, about 11 degrees; one of 0.08 goes before 5. Put the measured numbers into the simulation, run the plan again, and if it now fails on screen, fix the plan on screen, where fixing is cheap. Then try it on the real robot, because there is always a third number nobody thought to measure.

The other answer is for when you cannot measure, because the robot will meet a thousand different objects and nobody can weigh them all. Domain randomization is running the plan in many simulated worlds at once, each with its numbers shaken a little, grippiness anywhere from 0.05 to 0.3, say, and mass from thirty to a hundred and fifty grams, and accepting the plan only if it works in every one of them; a plan that fails in some is changed and tried again. A plan that survives a crowd of made-up worlds usually survives the real one, because the real one is somewhere in the crowd. This plan would have failed that test before the real robot ever touched a block: it survives every mass but no grippiness under 0.10. The engineer would have read that and reached for the fixes the module on grippers ended with, rubber pads or better-shaped jaws, without dropping anything real.

What the simulation leaves out

Both sides of this box are simulations, so both lie in the same ways, and the left side's drop is itself a prediction, not a recording. A real block would tumble out of the jaws a little differently every time, and a real jaw would flex. Notice, too, that pressing Play again gives exactly the same drop at exactly the same moment, which no real robot has ever done: the sameness that makes simulation so useful for finding things out is also one of its lies.

Another is in the box itself. The Grippiness slider cannot make the model slipperier than the real block, though a model can be wrong that way too: below a grippiness of about 0.06 this simulation's arithmetic for touching surfaces breaks down and flings a pinched block across the room. The numbers this course has quoted all along, the 13-degree stall, the line at a grippiness of 0.10, are this simulation's numbers. A real arm of this design would have its own, a little different, and finding out how different is where the work on a real robot begins.