42Where robotics is going, and where you go next

Forty-one modules ago a robot vacuum met a chair leg and turned. Everything since has been about the three things it did in that moment: it noticed, it decided, and it moved.

This page is unlike every other in the course. Nothing new is taught here, and the box below has no lesson in it. It is every robot you have met, one at a time, with the controls you know and a line saying what to do with each. The words are a look back at what you have seen, an honest look at where robotics is going, and a short list of things to read and do when you leave.

One loop, forty-one times

The first module gave the vacuum's moment a name, the sense-think-act loop, and said that a robot is a machine that runs it on its own: take in what the sensors report, decide what to do, do it, and go round again. Every module since has been about one stage of that loop, or about the loop as a whole. Here it is with the course laid on it.

1. SenseTake in what the sensors report, and be a little wrong about all of it.How robots sense
3. ThinkDecide what to do: by a rule, a plan, or something learned.How robots are controlledHow robots thinkHow robots learn
5. ActMove: a motor turns a joint or a wheel, and the world changes.How robots move
7. And round again, many times a second. These parts were about the loop itself: what it is, how it is written down, and where it goes wrong.What a robot isHow robots are programmed

Sense. The sensors you met measure one simple thing each. A distance sensor measures a straight line to the nearest thing ahead. An encoder counts how far a joint has turned, in ticks. A touch sensor reports a force. A camera hands over a grid of colored squares, and a recognizer turns the grid into a word. Every one of them was wrong by a little: readings jitter, an average of them arrives late, and wheels counted turns the floor did not match.

Think. Three kinds of deciding, in the order the course met them. A rule: measure the gap between where you are and where you want to be, and push in proportion. A plan: a map, a cloud of guesses about where you are on it, a search for a way through, a machine of states that says what to do next. And something learned: a motion copied from demonstrations, a table of pushes corrected by points, a network fitted to pictures. The learned ones are the newest, and the least finished.

Act. Joints that turn and joints that slide, a hand whose place is the sum of its joints, motors that trade speed for force and sag under a load they cannot hold, wheels that turn the robot by running at different speeds, four legs on a canned trot, four propellers.

And round again. Two parts were about the loop itself: what makes a machine a robot, and how the loop is written down: as a program, as small programs passing messages, in a simulation that lies a little about friction, and with safeguards between a fast arm and a person.

The robots, with nothing to learn

Seven robots in one box. Choose one with the buttons above it; the line under them says what to do. The ones that stand still are on sliders, the ones that move are on the arrow keys or the stick, and the cart-pole stands under its controller with two knocks and Run.

Each of them also has a way of failing that shows what is inside it. The three tries below bring out three of those, and the section after them compares what you saw.

The same three robots, compared

All three have motors that are told what to do. What differs is what, if anything, is doing the telling.

The rover has two motors, one for each wheel. In this box the stick or the arrow keys tell each wheel a speed, and that is all that happens inside it. No sensor is read and nothing decides, so at the wall it does not know there is a wall, and it keeps pushing. In the first module the same rover had a short program that read its distance sensors and steered; here that program is gone and you are doing its job. A rover with no program is a remote-controlled car.

The drone has four motors, one for each propeller, and it cannot hover without a program. Many times a second a program reads how the body is tilted and how fast it is climbing, and sets all four motor speeds to bring it level and hold its height. That is the whole loop, sensing, deciding and acting, done by the drone itself. When you cut a motor, the program goes on reading and goes on setting four speeds, but only three motors answer. Each propeller also twists the body a little, and the four are paired so the twists cancel; with one gone they no longer cancel, so the body spins, and with three propellers' lift it drops. It is on the floor within half a second.

So the drone has what the rover lacks, a program reading sensors and deciding. But that program was written for a drone with four working motors. It has no sensor for a missing one, so it goes on controlling a machine that is no longer there.

The cart-pole has one motor. It pushes the cart along the rail, and the most it can push with is 12 newtons. The pole sits on a free hinge with no motor at all. A program reads the pole's lean and the cart's position many times a second and pushes the cart under the pole: the same loop the drone has. The soft knock leans the pole about 8 degrees and it is caught. The hard knock leans it further and faster. The further a pole leans, the harder the cart must be pushed to get under it in time, and past about 27 degrees that is more than 12 newtons can do. The motor gives everything it has, the cart runs to the end of the rail, and the pole falls anyway.

So the drone and the cart-pole both have a program deciding, and both fail, for different reasons. The drone's program was given a machine it was not written for. The cart-pole's program asked for more push than its motor has. In neither case did the program decide wrongly.

Those are the three ways every robot in this course stopped working. Nothing was deciding, as with the rover. The program's picture of the machine was wrong, as with the drone, and as with the module 36 arm that was told its block was grippier than it was. Or the motors had no more to give, as with the cart-pole. The dog in this box fails the first way: nothing in its walk reads its tilt, so a small shove its stiff legs hold and a hard one topples it. When you watch a robot video, ask which of the three would stop that robot, and whether the film shows it.

Where robotics is going

Four things are changing, and each of them is something you have already handled.

The think step is being learned instead of written. For most of this course a person wrote the think step, as a rule or a plan, and in Part 6 nobody did: the robot was shown, or paid, or fitted to pictures. That is the change behind the robots in the news, folding laundry and tidying tables on a spoken request. What it runs on is recorded motion, and there is not much of it. The text a language model learns from was already written, by a great many people, for other reasons. Every demonstration a robot learns from is made on purpose, by a person driving the robot through one job. That is a large part of why the newest robots still fail often enough that a person stands near them, and why so much of the work now is collecting more.

Practice is moving into simulation. The learner in module 39 ran 400 tries in a few seconds of your time and 40 minutes of its own, and a real cart would have taken the 40 minutes. So slow or dangerous practice is increasingly done in worlds like these boxes, thousands at once, and the result is carried to the real machine. Module 36 is the catch: the simulation lies about friction and mass, and a grip learned on a block that was too grippy drops the real one. Closing that gap is a field of its own.

Language is becoming the way in. Understanding a request in nearly any wording is real, and a few years old, as module 41 showed. Doing it is still decided after the understanding, by whether the robot has a skill for what was asked. The arm there understood "pick up the egg" perfectly and had one squeeze, and the egg cracked.

The hardware is becoming open. The arm you have been driving is a real design whose drawings are published for anyone to build, and the dog and the drone are machines on sale today; their models came from a public library of robot models kept by a research laboratory.

Two things have not moved much. Hands: picking up an egg gently and a block firmly with one hand, as you do without thinking, is still an open problem. And trust: a robot near people must have its ways of failing found and guarded against before it is allowed there, and nobody can read off a learned think step's numbers when it will fail. That is one reason robots are in warehouses and factories before they are in kitchens.

Where you go next

Everything here is free to read or run, each line says what it assumes, and the one thing you could buy is marked as something you do not need.

If you want the mathematics this course left out, read Modern Robotics by Kevin Lynch and Frank Park, which is free to read online. It begins where this course stopped: its early chapters build up to the position of the hand, the sum you did on the planar arm, written as matrices. Expect to work.

If you can write code and want the real thing behind the node graph of module 35, do the ROS 2 tutorials. They run on your own computer with a simulated robot, in Python or C++, starting with nodes and topics as the box drew them.

If you want to keep playing with the physics in these boxes, the engine under every one of them is MuJoCo, free and open source, and the robot models came from its Menagerie. Its viewer opens a model with no code at all.

If you want a robot you can touch, and you do not need one, the arm in this course is the SO-101: open hardware, printed parts and hobby servos, with the drawings published. Hugging Face's LeRobot software calls it their flagship robot and documents building it, and their free robotics course goes from classical robotics to the learning of Part 6 on that software. It asks for basic Python.

Deeper courses on the things this survey only touched are being written here. When they exist, this page will say where.

The vacuum meets the chair leg and turns. You know now what happened in that moment, where each piece of it comes from, and where each of them gives out. That is how a roboticist looks at a machine. The rest is practice.