6Frames: where "left" is depends on who you ask

Sit across a table from a friend and put a cup between you, a little off to one side. Ask where it is. Your friend says "in front of me, a bit to the left." You say "in front of me, a bit to the right." You are both right, and you are both describing the same cup. The cup did not move; what changed was who it was measured from. Every position works this way: "in front, to the left" only means something once you know from where, and facing which way.

What a frame is

A frame is a point to count from and three directions to count along, usually forward, left and up. A position is then three numbers: so far ahead of the point, so far to its left, so far above it. Behind, right and down are the same three numbers gone negative. Change the frame and the numbers change, though the place does not.

You have already used one. In the last module every position was "x forward of the base, z above the floor". That was the base's frame: the point was the spot where the base is bolted down, and the directions were the way the base faces and straight up. Nothing was ever measured from anywhere else, so the frame never needed a name. On a robot bolted to a table, the base's frame is the one thing that never moves, and robot makers call it the world frame: the frame everything around the robot is described in.

But there is nothing special about the base. Any rigid part can carry a frame: pick a point on the forearm and three directions fixed to it, and wherever the forearm goes, they go with it. The frame that matters most rides on the hand, with its forward wherever the hand is pointing right now. Robot makers call it the tool frame, because the hand is usually a tool: a gripper, a welding torch, a suction cup.

Why a robot cares

Here is the trouble frames cause. The arm in the box below is a real one, the SO-101, a small arm with a gripper that people build at home, and it has a bracket on its wrist for a small camera. That camera sees the world in the hand's frame: a cup shows up as "20 cm ahead of the hand, 5 cm to the hand's left". But moving the arm means finding joint angles that put the hand on the cup, which is the next module's problem, and however the robot goes about it, the only way it can check an answer is the working from the last module, which starts at the bolted-down base and answers in the base's frame. So the cup has to be in the base's numbers too, or there is nothing to check against. The camera answers in one frame; the joints need another.

So the robot converts, with a tool you already own. Forward kinematics told you where the hand is in the base's frame. It also tells you which way the hand's three arrows point. Picture them painted on the hand. The working walks out from the base, link by link, and turns at every joint on the way; whatever is painted on the hand turns with every one of those turns, so when the working arrives at the hand it knows where each arrow has ended up. On the flat arm every joint turned about the same line, so one number, the tilt, said it all. On this arm the joints turn about different lines, a sideways turn at the shoulder and a roll at the wrist among them, and the same walk delivers three arrows pointing three ways. Where the hand's frame sits and which way each arrow points, in the base's frame, is exactly what forward kinematics works out. Knowing that, "20 cm ahead of the hand and 5 to its left" becomes a position in the base's frame by the same kind of walk: start at the hand's point, step 20 cm along the hand's forward arrow, then 5 cm along its left arrow, and read off where you are. The robot does this conversion many times a second, for every camera, every force sensor and every command, because almost nothing on a robot is measured in the frame it is needed in.

Try it

Unlike the flat arm of the last module, the SO-101 works in all three directions, so it has two joints at the shoulder and two at the wrist, one slider each. Shoulder turn swings the whole arm left and right. Shoulder lift tips it forward and back. Elbow bends it. Wrist bend tips the hand up and down. Wrist twist rolls the hand around its own length, like turning a key. Gripper opens and closes the jaws.

Three of the arm's frames are drawn on it. Each is a red arrow for forward, a green arrow for left and a blue one for up. The base's frame sits on the floor under the base, and its red arrow points the way the arm faces at the start. The elbow's frame sits at the elbow joint, on the forearm, with its forward running down the forearm toward the wrist. The hand's frame sits at the wrist-twist joint, at the back of the gripper, with its forward running out along the jaws. A star floats in front of the arm, and the Frames button hides the arrows if they get in the way.

Beside each slider is its readout in degrees, as before. Under the sliders, "Measured from" picks a frame, and two lines give the hand's position and the star's position as that frame sees them, each number in the color of the arrow it was measured along. One warning: "left" here is the arm's left, along its green arrow, which may not be left on your screen. Your screen is a fourth frame, and that is the whole point.

Loading the simulation. The robot runs in your browser, so the first one takes a few seconds to arrive.

ResetPauseSlowLabels

This is easier on a larger screen, but it works here too.

What you saw

Measured from the base, the hand started 16 cm ahead, neither left nor right, and 14 cm up, and turning the shoulder swung it to the left: forward kinematics as before, in the base's frame, with a left column added because this arm has a joint that turns sideways. Measured from the hand, the hand was at 0, 0, 0, and stayed there whatever you did. That is not a trick. Ask the hand where the hand is and the only honest answer is "here", which is why a robot only ever asks where the hand is in some other frame.

The elbow's frame shows what a frame on a moving part is for, if you choose Elbow and move the shoulder and elbow sliders. Turning the shoulder, lifting it, or bending the elbow carries the hand around the room but does not change its numbers at all, because the elbow's frame rides on the forearm, and those three joints carry the forearm and the hand along together, so the two never move relative to each other. Only the wrist bend, which sits between the elbow's frame and the hand, changes the reading. A position measured in a frame only cares about what happens between that frame and the thing being measured.

Rolling the wrist twist was the sharpest one. Neither the hand's point nor the star moved, yet the star's numbers changed, left trading places with up as the hand rolled a quarter turn. All that changed was which way the hand's arrows point, and turning a frame's directions is enough to change every number measured in it. A camera on that wrist would have watched the star slide across its picture while nothing in the room moved.

In the last try you steered by the camera's kind of numbers alone: the Star line said which way the star lay in the hand's own terms, and you nudged the joints until it read zero. That works, one nudge at a time, and it is how a person drives an arm by hand. A robot that wants to get there in one move cannot nudge and look; it needs the star in the base's numbers, so that it can find and check the angles that take the hand straight there. Switching to Base at the end showed you what the conversion delivers: 20 cm ahead, 10 cm left, 12 cm up, the star's one address in the frame that never moves, the same whatever the hand was doing.

Two ways to drive a real arm

Robot arms in factories are moved by hand with a pendant, a handset with a button for each direction, and the pendant offers a choice of frame. In world mode, the "forward" button moves the hand along the world frame's forward, however the hand happens to be tilted. In tool mode, the same button moves the hand along its own forward: straight into the part it is working on, from whatever angle the tool is pointing, which is what you want when lining up a drill. Most pendants also let the operator set up a frame of their own, lined up with a tilted table, say, so that "forward" runs along the slope. The buttons are the same. What changes is who you ask where "forward" is.