Reward and Punishment

Reward and Punishment

The iron rule of nature is: you get what you reward.

If you want ants to come, you put sugar on the floor. Charles Munger

John's actions brought him praise and money, making him continue his behavior. We do what is rewarding and avoid what we are punished for. We learn from the consequences of our actions, though a reward does not make an action right. Whatever causes us to repeat a certain behavior is reinforcing and whatever makes us stop is punishing. Behavior that is rewarding or feels pleasurable tends to be repeated. A consequence that people want can encourage repetition; an unwanted consequence can discourage it. Neither effect is automatic: people may persist despite punishment or decline a reward that does not matter to them. In the beginning, rewarding (or punishing) people is most effective when it is administered without delay and each time the behavior is repeated.

Once a behavior becomes learned, variable rewards strengthen the behavior. Some unpredictable reward schedules can sustain high response rates and make behavior difficult to extinguish. This is, for example, how gamblers are rewarded. When they don't know when the reward will arrive, they try again and again. The size of a reward can also affect persistence.

We base what is rewarding or punishing on our associations to past experiences and their consequences or with values or preferences we are born with.

An action that is reinforced becomes stronger over time. Reinforcement can contribute to habits, superstitions and addictions, all of which can be hard to change. As the great 18th Century English writer Samuel Johnson said: "The chains of habit are too weak to be felt until they are too strong to be broken."

What does all this mean? It means that people do what works. Like bees to honey, they go where there is a reward. This also implies that if we reward what we don't want, we get it. As Garrett Hardin says in The Ostrich Factor: "If the laws of society reward for bank robbing, society will get more bank robbing. If our methods of winnowing candidates for high positions favor stupidity, we will get stupid politicians."

Studies in Sweden show that changes in the sickness insurance system affect sick leave behavior. Reforms implying more generous compensation for sick leave tend to be associated with permanent increases in total sick leave per person employed. Other studies from the U.S. regarding health services have shown that in cases where someone else picks up the costs, patients tend to over-consume the health services.

Lower personal costs can encourage greater use of a service. Greater use does not by itself establish abuse or unnecessary consumption. The incentive problem arises when people receive a private benefit from misuse while sharing its costs with others. The more people that benefit from misusing the system, the less likely it is that anyone will draw attention to what really happens. Individually they get a large benefit and it's a small loss for society. Until everyone starts thinking the same. The ancient Greek philosopher Aristotle said: "That which is common to the greatest number gets the least amount of care. Men pay most attention to what is their own; they care less for what is common."

Why do people steal?

The account cites a survey in which 23% of respondents said they would steal if they could not get caught. It also cites a contemporary estimate of $400 billion in annual U.S. business fraud losses, about a third attributed to employee theft. Why? Opportunity and reward. In Charles Munger's words: ''The worst abuses come where people have the greatest temptations." Making theft easy increases the opportunity for abuse, and tolerated misconduct can spread.

Charles Munger tells us how a bad policy can become the norm:

In the New York Police Department, they have a simple system. Your pension is based on your pay in your final year. So when anyone reaches the final year, everybody cooperates to give him about 1,000 hours of overtime. And he retires - in some cases after a mere 20 years of service - with this large income. Well, of course his fellow employees help him cheat the system. In substance, that's what's happened. But the one thing I guarantee you is that nobody has the least sense of shame. They soon get the feeling they're entitled to do it. Everybody did it before, everybody's doing it now - so they just keep doing it.

Why does a tennis player always wear his lucky shirt in the finals?

In one experiment, American psychologist B.F. Skinner fed pigeons small quantities of food at regular intervals. After some time the pigeons began to behave superstitiously. If a pigeon was bobbing its head when food appeared, it got the idea that bobbing its head must have made the food appear. The pigeons repeated movements that had coincided with food arriving, even though food appeared independently of what they did. The apparent connection was accidental. Skinner wrote in "Superstition" in the pigeon: "There are many analogies in human behavior... A few accidental connections between a ritual and favorable consequences suffice to set up and maintain the behavior in spite of many nonreinforced instances."

Are the right incentives important?

Incentives act as reinforcers. Charles Munger tells a story about the importance of getting the incentives right:

From all business, my favorite case on incentives is Federal Express. The heart and soul of their system -which creates the integrity of the product- is having all their airplanes come to one place in the middle of the night and shift all the packages from plane to plane. If there are delays, the whole operation can't deliver a product full of integrity to Federal Express customers. And it was always screwed up. They could never get it done on time. They tried everything- moral suasion, threats, you name it. And nothing worked. Finally, somebody got the idea to pay all these people not so much an hour, but so much a shift and when it's all done, they can all go home. Well, their problems cleared up overnight.

John invested in a biotech start-up that went sour and he lost money.

After a success, we may become overly optimistic risk-takers. After a failure, we may become overly pessimistic and risk-averse - even in cases where success or failure was merely a result of chance. Good consequences don't necessarily mean we made a good decision, and bad consequences don't necessarily mean we made a bad decision.

The next time someone presents John an investment opportunity in a biotech start-up the chances are he will decline. He associates the new proposal to his earlier experience. And since people tend to believe that the future mirrors the past, he declines. But what happens if John's first investment made him a lot of money? Wouldn't John associate the new proposal to his old pleasurable experience? Isn't he therefore more likely to invest? This automatic association to what worked in the past causes people to under-react to new conditions and circumstances.

Mark Twain understood the dangers of blindly trusting past experience for dealing with the future. "We should be careful to get out of an experience only the wisdom in it, and stop there, lest we be like the cat that sits down on a hot stove-lid. She will not sit down on a hot stove-lid again - but also she will never sit down on a cold one anymore."

Does it matter in which size or order rewards or punishments arrive?

Mary never wraps the kids’ Christmas presents in one box.

Since our experiences seem longer when broken into segments, we like to have pleasurable experiences broken into segments but painful ones combined. That is why Mary puts presents in many boxes. Frequent rewards feel better. For example, it feels better to gain $50 twice than $100 once since every gain is rewarding. It feels better to lose $100 once than $50 twice since every loss is painful.

We prefer a sequence of experiences that improve over time. Losing $100 first and then gaining $50 seems more rewarding than gaining $50 and then losing $100. We want to get rid of the bad experiences first. Immediate losses are preferred over delayed ones. Just as we don't like bad experiences, we also don't like waiting for them. We like to get over them fast.

Mere association and reinforcement are both examples of conditioned reflexes. Charles Munger gives an example how these forces can be used as he describes the invention of non-alcoholic drinks:

The food value and so forth of the beverage is the reinforcer. And the trade dress, trade name and look of the beverage is the stimulant... Then, you go on to the second kind of conditioned reflex- and that's straight Pavlov... Well, how do you get Pavlovian mere-association effects? Obviously, you associate this beverage and its trademarks with every good thing that people like generally: exalting events, sex objects, happy times - you name it.

How can we lose a conditioned reflex that's working for us? Charles Munger continues:

Well, the customer tries something else and discovers that it's a big reinforcer. So he shifts brands. We know, in matrimony, that if you're always available, the spouse is less likely to shift brands. And people don't tend to organize marriage to include permanent long separations. Similarly, if you're selling a product and it's always available, people are less likely to shift to some other product and get reinforced by it.

Keep in mind

• American statesman, scientist and philosopher Benjamin Franklin tells us that: ''A spoonful of honey will catch more flies than a gallon of vinegar." Praise is more effective in changing behavior than punishment. It is better to encourage what is right than to criticize what is wrong.
• Set examples. Michel de Montaigne said: "It is a custom of our justice to punish some as a warning to others. For to punish them for having done wrong would, as Plato says, be stupid: what is done cannot be undone. The intention is to stop them from repeating the same mistake or to make others avoid their error. We do not improve the man we hang: we improve others by him."
• Don't over-learn from your own or others’ bad or good experiences. The same action under other conditions may cause different consequences.
• Distinguish skill from chance. Charles Munger says, "As you occupy some high-profit niche in a competitive order, you must know how much of your present prosperity is caused by talents and momentum assuring success in new activities, and how much merely reflects the good fortune of being in your present niche."
• American novelist Upton Sinclair said: "It is difficult to get a man to understand something when his salary depends upon him not understanding it." Since people do what works, be sure to make the incentives right. Tie incentives to performance and to the factors that determine the result you want to achieve. Make people share both the upside and downside. And make them understand the link between their performance, their reward and what you finally want to accomplish. For example, tie a manager's compensation to gain in business value (of the unit under his control) minus a cost factor for capital that is used to produce this value. The auto insurer GEICO's plan exemplifies Berkshire Hathaway’s incentive compensation principles. Warren Buffett says:

Goals should be (1) tailored to the economics of the specific operating business; (2) simple in character so that the degree to which they are being realized can be easily measured; and (3) directly related to the daily activities of plan participants. As a corollary, we shun "lottery ticket" arrangements, such as options on Berkshire shares, whose ultimate value

• which could range from zero to huge - is totally out of the control of the person whose behavior we would like to affect. In our view, a system that produces quixotic payoffs will not only be wasteful for owners but may actually discourage the focused behavior we value in managers.

• Reward individual performance and not effort or length in organization and reward people after and not before their performance.
• Don't let money be the only motivation. If we reward people for doing what they like to do anyway, we sometimes turn what they enjoy doing into work. The reward changes their perception. Instead of doing something because they enjoy doing it, they now do it because they are being paid. The key is what a reward implies. A reward for our achievements makes us feel that we are good at something thereby increasing our motivation. But a reward that feels controlling and makes us feel that we are only doing it because we're paid to do it, decreases the appeal. Blaise Pascal said: "We are generally the better persuaded by the reasons we discover ourselves than by those given to us by others."
• Install systems and rules that encourage the behavior you want. Never let it pay someone to behave in a way you don't want. Have systems that make it hard for people to get away with undesirable behavior. Make undesirable behavior costly. The painful consequences of undesirable behavior must outweigh its pleasurable consequences. For example, the consequences of spending time in jail ought to be more painful than the pleasure of getting away with burglary.
• Systems can be hard to change as Warren Buffett observes, "It's very hard to change a system when the guy whose hand is on the switch benefits enormously and, perhaps, disproportionately from that system."
• Decision-makers should be held accountable for the consequences of their actions. In The Case for Modern Man, American philosopher Charles Frankel defines responsibility: ''A decision is responsible when the man or group that makes it has to answer for it to those who are directly or indirectly affected by it." Charles Munger adds, ''An example of a really responsible system is the system the Romans used when they built an arch. The guy who created the arch stood under it as the scaffolding was removed. It's like packing your own parachute."