35How real robot software is organized: nodes and messages

The last module's program did everything in one loop: read the bumper, read the rangefinder, decide, drive, and round again five times a second. Thirteen lines, and it was the whole robot. Now count what the rover has carried through this course: five rangefinders, a bumper, a counter on each wheel, the sum that turns wheel turns into a position, a brain that dodges, the wheels themselves, and on a robot with a map, a piece that works out a route across it, which can take a second of thinking. Each of those wants a different pace: the route can wait a second, the wheels need telling many times a second. Written as one loop, every piece would wait for the slowest, because a loop does not interrupt its own round. This module is about how real robot software is cut up instead, and the rover is here as a drawing of its own software rather than a thing in a pen.

Cutting the loop up

Take the last module's program and cut it into pieces by job. One piece reads the rangefinders, five times a second, and does nothing else. One reads the bumper. One takes the distances and the bumps and decides on wheel speeds. One drives the motors. Each piece is a node: one small program that does one job and runs on its own, at its own pace, beside the others. The rover's software is all of its nodes running at once, tied together only by what they send each other.

What they send is a message: one packet of information, such as five distances or a pair of wheel speeds, made whenever the sending node has something new. Messages travel on a topic, a named channel: the rangefinder node sends its readings on a topic called /distances, and the node that decides on speeds listens to /distances. The names start with a slash by habit, as file paths do. A node sends on a topic without knowing who is listening, and a node listens to a topic without the sender knowing it is there. The topic's name is the only thing the two have to agree on.

How does a message find its listener, then, if the sender does not know who that is? By introductions made at the start. When a node starts, it announces which topics it will send on and which it wants to hear; the announcements are matched up, each sender to each listener of the same topic, and from then on messages go straight from one to the other. The sender hands its message over; who it reaches was settled by the matching, not by the sender. Think of a notice board with a heading for every topic: you pin your note under /distances and walk away, and whoever is watching that heading takes a copy.

That one rule, nobody knows who is listening, is the whole trick. The rangefinder node is written once and sends distances; whether one node uses them, or three, or none, it sends the same way. A new node can be added that listens to a topic and nothing already running has to change. A node can be taken away and the ones that sent to it go on sending, into the air. And the piece that decides can be swapped for a better one over lunch, so long as it listens to the same topics and sends on the same ones.

What one node does

A node that reads a sensor runs on a clock of its own: the rangefinder node reads the five beams five times a second and sends a message each time, whatever else is going on. That is the last module's loop rate, moved into the node that owns the sensor. Every other node here is woken by a message: the node that decides on speeds does nothing until a message reaches it on /distances or /bump, then reads it, works out the two wheel speeds, sends them on /speeds, and goes back to waiting. The motors node waits on /speeds and sets the wheels each time one arrives. The bumper node sends five times a second too, so the deciding node is woken ten times a second, and the motors are told ten times a second.

The nodes that make the robot move are not the only ones. One node listens to /pose, where the rover believes it is, and /speeds, and writes every message it hears to a file with the time, and sends nothing. That is a recorder. Nobody sent it anything; it listened. When the rover does something odd on a Tuesday afternoon, the file is how an engineer finds out what the sensors said and what the brain decided, message by message.

The Robot Operating System

Nodes need a way to find each other's topics, a way to pack a message so the listener can unpack it, and tools for looking at what is flowing. Most robots built in the last fifteen years get all three from one open-source system called the Robot Operating System, written ROS. The name is misleading: it is not an operating system in the sense that Windows is, and it runs on top of an ordinary one. It is the plumbing between nodes and the habits that go with it, plus a tool that draws the running nodes and their topics as a picture; the picture in the box is drawn the same way, though nothing in the box runs ROS. On a robot running ROS the topics have names like /scan for a rangefinder's sweep, /odom for where the wheels say the robot is, and /cmd_vel for the speeds sent to the motors; the box uses plainer names for the same things.

The box

The picture is the rover's software, seven nodes, each a box with its name. On the left are three nodes that read a sensor, each with its pace written under its name: the wheel counters from the odometry module, ten times a second, and the rangefinders and the bumper, five times a second each. In the middle, Odometry listens to /wheel_turns and sends the rover's believed position and heading on /pose, and the Avoider, the last module's brain, listens to /distances and /bump and sends on /speeds. On the right, Motors listens to /speeds, and the Recorder listens to /pose and /speeds and sends nothing. Each line is a topic with its name on it, running from the node that sends to the node that listens, so everything flows left to right; every message on its way is an orange dot sliding along the line that way, slowed down to six tenths of a second a line so that you can watch it; a node that has just received one gets an orange ring. The small rover in the corner drives round in a circle while the Motors node is being fed, and stands still when it is not.

Under the picture is a Stop or Start button for every node; a stopped node turns gray and neither receives nor sends, and Reset starts them all. Below the buttons is a table of every topic and how many messages a second are sent on it, and under that a line saying how many messages are on their way, which nodes are stopped, and whether the rover is driving. The box opens running.

ResetPauseSlowLabels

Stop Wheel countersStop RangefindersStop BumperStop OdometryStop AvoiderStop RecorderStop Motors

Topic	Messages a second

Waiting for the first frame…

This is easier on a larger screen, but it works here too.

What you saw

Every stop you pressed cut a graph at one node, and what went quiet was exactly what was downstream of the cut and nothing else. Stop the Avoider and the three sensors kept sending at their own pace, because a sensor node's clock is its own and its job does not include checking whether anyone is listening. Stop the Motors and the Avoider kept deciding, into a gray box. Stop the Recorder and the rest of the robot did not change by one message a second, because the Recorder had only ever listened. The arrows in the picture are the only dependence there is.

The second try is the one to remember. With the rangefinders stopped, nothing crashed, nothing went quiet except one topic, and the rover drove on, deciding its speeds from bumps alone. A robot made of nodes fails in pieces, which is good when the piece is the recorder and bad when the piece is the eyes; the difference between the two is invisible in the picture and plain in the table. Engineers who run robots this way watch the rates the way a nurse watches a pulse: a topic that should carry five messages a second and carries none is the first sign that something upstream has died, and a node that notices and stops the wheels is a standard part of the software.

The rover stopped about a second after the Avoider did because the last speeds were still sliding along their lines; that second is the box's slowed-down dots, not a robot's, and the order of events is the robot's.

Why cut it up

Four reasons, and the tries showed three. Each job runs at its own pace, so the planner can think for a second while the wheel counters send ten times a second and nobody waits for anybody. A node that crashes takes down its own job and no other, and a watching node can stop the wheels while the rest is restarted. A node can be added, like the recorder, or replaced, like a better avoider, without touching the rest, so a team of ten can build a robot by agreeing on topic names and each writing their nodes. And a recording of every topic can be played back into the software later, with the real sensors switched off, so a mistake made on a Tuesday can be studied on a Wednesday.

Every robot in this course that did several things at once was built this way underneath: the first module's vacuum has a node for its bumper, one for its distance sensors, one for the rules that decide, one for the dock's signal and one for the wheels. If you came through the modules on brains, each brain you met, the rules that pick a state or the sum of pushes that dodges, lives inside one node. The sense-think-act loop of the first module is what the whole graph does, spread across many nodes, each a piece of sense, think or act.

What the simulation leaves out

The dots here carry nothing; a real message carries numbers, five distances or two speeds, and the box only shows that one was sent. Travel here takes six tenths of a second a line so the flow can be watched; on a real robot it takes well under a thousandth, and a message is late because its sender was busy, not because the wire was slow. Every node here that is woken by a message sends one straight back, so ten messages a second in means ten out; a real avoider might send only when the speeds change, and a real planner takes its time. And a node here stops because you pressed a button. On a real robot a node stops because it crashed, and the first anyone knows of it is a rate in a table falling to zero.