24Stability: when control goes wrong

Stand a broom on your open hand, bristles up, and try to keep it there. You watch the top. When it starts to lean left, you move your hand left, under it, and it comes back. You do not have to be quick, because a broom falls slowly. Now imagine doing it through a video call with a lag of half a second. You see the broom lean left and move your hand left. But that was half a second ago. By now the broom has come back on its own, or is leaning the other way, and your hand arrives exactly where it should not be. The broom is down in a few seconds, and you were doing everything right, just late.

This module is about that lateness. The box holds a robot built for it, and one slider.

The cart and the pole

The robot is a cart-pole, and it is new to this course: a cart on a rail with a pole standing on it, kept up only by the cart moving underneath. A small cart runs along a rail, two meters long with a bumper at each end. One motor pushes the cart left or right along the rail with up to 12 newtons of force, about the weight of a bag of sugar, and that is the only thing the motor can do. A pole half a meter long stands on the cart's side, on a pivot that turns freely. Nothing holds the pole up. Left alone it falls over in about a second and hangs below the cart, swinging, like a pendulum. A sensor on the pivot reports which way the pole leans and how fast.

The controller keeps the pole up the only way it can: by moving the cart. Its rule is the last module's P and D, aimed at the pole. When the pole leans to the right, push the cart to the right, under it, and harder the further it leans. Brake that push by how fast the pole is going over, so the catch does not throw it the other way. Two more pushes, smaller, keep the cart near the middle of the rail, because a cart that is always catching the pole drifts along it and would meet a bumper. In this box the sizes of those pushes are fixed.

That rule has one strange thing in it, so look at it before you go on. To catch a pole that leans right, the cart goes right. That is what your hand did under the broom: it went under the lean, not away from it. And to bring the cart back to the middle when it has drifted right, the controller first pushes it a little further right, so that the pole leans left, and then the lean carries the cart left under it. A controller that pulled the cart straight toward the middle would leave the pole behind and tip it over.

Old news

In the last three modules the controller saw the joint's exact angle at the instant it acted. No real controller does. The sensor, the wire to the computer, the computer working out the push and the motor delivering it each take a moment. Engineers call the whole gap latency: the time between something happening and the controller acting on it. In a good servo it is a few thousandths of a second; through a wireless link, a camera or a slow program it can be a tenth of a second or more.

The slider in the box is that gap. At 0 the controller sees the pole as it is now. At 60 milliseconds it sees the pole as it was 60 milliseconds ago, and acts on that. Nothing else changes: the same pushes, the same sizes, the same motor.

The box

The box holds the cart-pole with the pole standing and the controller running. Run puts the pole back up with the cart at the middle and starts the controller afresh, the same way every time; Knock the pole gives the pole the same flick every time. The upper chart draws the pole's lean over the last few seconds, degrees either side of upright; the lower one draws the force the motor is giving. The line under the charts says which way the pole leans, where the cart is and, once the pole is still, the biggest lean since the last knock. And in this box the controller gives up once the pole has leaned past 45 degrees, far beyond anything it could catch, and switches itself off so that you can watch the pole fall and hang; the line under the charts says so.

Loading the simulation. The robot runs in your browser, so the first one takes a few seconds to arrive.

This is easier on a larger screen, but it works here too.

That is a knock being caught. The pole swung to the right, back past upright to the left, and back again, each swing smaller than the one before, until nothing was left of it. Swinging back and forth past the target like that, again and again, is called oscillation, and you have seen it before: the arm arriving at its star in the last two modules did it. An oscillation that dies away is the sign of a controller that is winning. A system that behaves like this, where a knock fades on its own and it comes back to where it was, is stable.

Now start making the news late. At 40 milliseconds a knock leans the pole further, about 13 degrees, and it rocks a few more times before it settles. It is still caught, but it was closer.

Why late is worse than weak

Nothing about the controller got weaker between 50 and 60 milliseconds. It pushed exactly as hard for exactly the lean it saw. What changed is when the push landed.

Picture one swing. The pole leans right and starts back toward upright, because the cart has already gone under it. With no delay the controller sees the lean shrinking and eases off, and the brake, which reads the pole's speed, leans against the swing as it comes back through upright. The swing arrives on the left smaller than it left the right. Now add the delay. The controller is still acting on the lean it saw a moment ago, when the pole was further right, so it keeps pushing the cart right after the pole has already turned round. That push arrives on a pole that is now swinging left, and it helps the swing along instead of holding it back. Even the brake works against you: it reads a speed from a moment ago, so it brakes hardest just after the pole has slowed down and eases off just as it speeds up.

The brake survives a small delay better than the push does. Its push points the same way for the whole of a swing, against the motion, so being a little late only trims it; the lean push changes direction at every turn of the swing, so being late turns a slice of it right round. Every swing, then, the late push adds a little energy and the brake still takes a little away. Below 50 milliseconds the brake wins and each swing is smaller than the last. Above it the late push wins and each swing is bigger than the last. A system that behaves like that, where a disturbance grows on its own until something falls, hits a limit or breaks, is unstable. There is no middle. Either the swings shrink or they grow, and the delay decides which.

At the edge, around 50 milliseconds, the two balance, which is why a knock there rocks the pole for so long: nothing is winning. And notice where the cart ended after the fall, against a bumper. Look at the lower chart: as the swings grew, the pushes grew with them until the motor was giving all it had, 12 newtons one way and then the other, the flat tops on the force trace. That is the saturation of the proportional control module, and it arrived because the swings were already too big to catch, not the other way round. A stronger motor would have bought a few more swings, not a cure. The cure is a shorter delay, or a gentler controller that leans on the news less hard.

The further past the edge, the faster it goes. At 60 milliseconds the swings grow by about a fifth each time and it takes six seconds to fall. Set the delay to 100 and press Run, and the pole is down in a second and a half, in three swings: the swings nearly triple each time.

An unstable system does not need a knock, either. It makes its own. Slide the delay back to 0, press Run, wait until the pole is standing still, and then slide the delay up to 60 while it stands. Nothing happens for a while, because there is nothing to grow. Then a wobble appears out of nothing you can see, the tiniest leftover of the last catch, and grows, and about twenty seconds later the pole falls.

This is everywhere

Every closed loop in this course, every controller that acts on what it measures, has an edge like this one, and you have been near it once already. The stiff arm of the proportional control module, with its gain at 40, oscillated at the star where the soft arm did not; give that stiff arm ten milliseconds of delay and it oscillates for good, where this pole shrugs off forty. What matters is not the delay by itself but the delay compared with how long a swing takes. This pole takes about a third of a second to swing from one side to the other, so 40 milliseconds is a small slice of a swing and the pushes land nearly on time; the stiff arm's swings were about three times quicker, so ten milliseconds was the same slice of one. Stiffness and lateness trade against each other. A robot with a slow sensor, a camera at thirty frames a second, say, has to be controlled gently; a robot with a fast one can be controlled hard. Faster sensors and faster computers do not make a robot stronger; they let it be stiff without shaking itself apart. The drone in the flying module is kept level by a loop like this one, hundreds of times a second, and it has the same edge.

What the simulation leaves out

A real controller has no give-up rule: it would keep pushing the cart at full force after a pole it could no longer catch, and slam it into the bumper. The pole's sensor here is exact; a real one jitters, and jitter through the brake is another source of pushes at the wrong moment. And the delay here is one clean number. In a real robot the latency comes and goes, a camera frame late here, a computer busy there, so a real controller is tuned to be stable with some margin, well back from the edge. The next module on this robot puts the four pushes on switches and shows what each one is for, and where the margin runs out.