SKILL OR LUCK
Michael Mauboussin on the 'Success Equation'
How do we know which of our
successes and failures can be attributed to either skill or luck?
That is the question that investment
strategist Michael J. Mauboussin explores in his book The Success Equation: Untangling Skill and Luck in Business,
Sports, and Investing. Wharton management professor Adam
M. Grant recently sat down with Mauboussin
to talk about the paradox of skill, the conditions for luck and how to avoid
overconfidence.
An edited transcript of the
conversation follows.
Adam M. Grant: Michael, we're delighted to have you here today to talk
about your book The Success Equation.... I would love to hear you speak
a little bit about this paradox of skill that you have discovered and the
relationship between luck and skill.
Mauboussin: Let me first tell you what the definition of the
"paradox of skill" is. Specifically, it says that in activities where
skill and luck define outcomes, as skill improves, luck becomes more important
in determining outcomes. [By that definition,] more skill means more luck,
which seems very paradoxical. It's not my idea. I learned about it from Stephen
Jay Gould, the very eminent biologist at Harvard. He talked about it in the
context of Ted Williams, the last player to hit over 400 in major league
baseball, which he did in 1941. Gould was wondering why no one has been able to
achieve over 400 since that time. He looked at [variables such as] maybe
because the guys play at night, or they travel too much. Really, none of those
things checked out. Then he said, maybe Williams is just this amazing player --
an immortal among mortals.... But if you look at every other sport, for
example, things measured against the clock -- there has been absolute
performance everywhere you look, so that doesn't seem to be the case. Then he
thought about it more carefully, and he realized the actual result is because
everyone's gotten better, and as a result, the standard deviation of skill has
actually narrowed. If you think about batting average for your season and your
player, some level of skill plus some level of luck gives you your outcome.
What's happened generally is that the standard deviation of skill has gone
down. Why? Because you're recruiting players from the world now, versus from
just parts of the United States. You're training better. You're coaching better
-- all those kinds of things....
The point is this paradox of skill.
We've seen the differential skill narrowing. We see it really all over the
place. We see it in the world of investing. We see it in the world of business.
I think it is very interesting. As skill improves, especially in competitive
markets, luck becomes more important determining outcomes.
Grant: It's incredibly interesting and much the opposite of what
most of us want to believe. [How do you apply this] as a business professional
or a leader?
Mauboussin: Well, a couple things are possible. One is to think about
finding fields where there is differential skill. If you see something that's
very highly competitive, you really need to have something completely different
to get you on the right side of the tail of the skill distribution. The easiest
thing is probably to think about things where there is differential skill or to
try to attack a more skillful player using an unusual tactic, for example.
Grant: That makes a lot of sense. Obviously, there are some
outliers who don't fit this picture. I think of Miguel Cabrera winning the
Triple Crown this past year. Is that luck? Or is there a way still to cultivate
skill, even though you're dependent on luck more than before?
Mauboussin: Athletics is a great example. One of the points I would
make, which I think is pretty common sense once I say it, is that whenever you
see an outlier in sports, it is always a combination of really good skill and
really good luck. One of the best ways we can measure that is through streaks.
For example, Joe DiMaggio had this 56-game hitting streak, but if you look at
all the players in major league baseball history who have had 30 or more
hitting streaks, their career batting average is over 300. They are about one
and a half or two standard deviations away from the average. To say it
differently, not all skilled players have streaks, but all streaks are held by
skillful players. You can almost be assured that whenever you see really,
really good results, it's skill and luck combined. By the way, you almost never
see it on the other side, which is really bad luck and really bad skill,
because those people either metaphorically or literally have to die off in the
population. We mostly see the outliers on the positive side versus the negative
side. We don't see the failures so much.
Grant: Are there ways that you can improve your luck?
Mauboussin: Maybe I should step back and define luck because I think
it's actually a fascinating topic. When you think about luck or read about it,
it really spills into philosophy, -- moral philosophy, very rapidly. By the
way, there are tons of aphorisms about luck. Luck is where success meets
preparation, and you make your own luck. Those aphorisms, while they have an
important sentiment, are actually not accurate. I'm going to define luck. Luck
exists when three conditions are in place. Number one is that it happens to an
individual or organization, so it could be you or your team or your company.
Second, it can be good or bad.... The third, and I think really essential
[condition], is that it's reasonable to expect a different outcome could have
occurred. When those three things are in place, there is luck. Now, by my
definition, another, simpler way to think about it is: What's in your control
versus what's out of your control? Luck would be what's out of your control.
When you hear people say, well, you make your own luck, what they're
encouraging you to do is work harder or be persistent or be gritty. Those are
all really important things. But if that's within your control, in a sense I'd
put that into the skill bucket.
How do you manage luck? I'll share a
couple of ideas. One is that there's a really simple heuristic that when you
are the favorite, the stronger player, you have positive asymmetric resources,
you want to simplify the game. When you're the underdog, you want to complicate
the game. That would be one example. Of course, the canonical example is David
versus Goliath, which is really a wonderful story if you've read the whole
story. David comes up, and he's delivering stuff to his brothers, and there's
this ruckus going on. What's going on? There's this guy, Goliath. He's a
6'5" dude, 130 pounds of armor, threatening everybody. David says,
"Yeah, I can take this guy on." Originally, they put him in armor.
He's going to go toe to toe with Goliath. He's this little skinny shepherd boy.
David gets this quickly. [He realizes that] this isn't going to work. He
immediately takes off all the armor. Of course, [he draws on] his famous sling
shot, and takes five stones from the creek. Then he goes out and uses his own
technique.
In business, that would be an
example of disruptive innovation. Rather than going straight at the leader, you
come at [him] with a flank strategy. In warfare, it would be guerilla
strategies versus, again, toe to toe. In football, it might be trick plays
versus running straight down the middle.... You [might] say it's common sense,
but it's remarkable in business and sports and even in military how
underutilized that strategy is.
Grant: That raises another interesting question. You're familiar
with the really robust evidence for overconfidence. David Dunning and his
colleagues have done some of the most interesting [studies]. About 90% of any
people you would ask would assume they are above average on any given attribute
-- intelligence, skill, and so on. What you're saying basically requires people
to assess whether they are a favorite or an underdog. We know people are biased
toward thinking that they're favorites. How do you temper that level of
overconfidence to make good judgments about how to change the rules of the
game?
Mauboussin: That's a really interesting [question]. One of the
frameworks I like [to use] is going back to the Daniel Kahneman-AmosTverskyidea
of inside versus outside view. You're very familiar with that. The inside view
says that when we're trying to solve a problem or tackle something, the typical
way that we do it -- and this is where overconfidence plays into this -- is
that we gather lots of information about the situation, we combine it with our
own inputs and then we project into the future. It's almost --
"idiosyncratic" might be strong, but it's your own point of view.
The outside view, by contrast, says,
"I'm going to look at the problem as an instance of a larger reference
class. I'm going to ask what happened when other people were in this situation
before." That's one of the ways to temper that overconfidence. Rather than
looking at this as my own unique situation where I think I'm above average, I'm
going to look at what's happened to everybody else who has tried this before.
Now, I actually am ambivalent about this argument on one level because if
you're an entrepreneur, we know that a high percentage of them are going to
fail. But we know that some small percent are going to succeed and create an
enormous amount of value for society and so forth. You want the entrepreneurs
to get out of bed in the morning and say, "I'm going to go take the
mountain." But if you step back and said, "Oh, statistically, that's
probably not a great [chance]" -- I'm a little bit ambivalent about the
argument, but that would be one of the ways to try to temper that.
The other thing I'll mention quickly
on this is the "under-sampling" of failure -- which is that people, a
company or team will pursue a particular strategy, and they'll succeed wildly.
Another team or company will pursue a very similar strategy and fail. But, of
course, the failures go away. What happens is you walk along and say, okay what
strategy works? You see that strategy and you see success and so you say,
"Oh, that's got to be great." You under-sample failure. That's
another way to temper some of the thinking and say, "I want to understand
the entirety of what's happened with this strategy," for example. Those
might be some ideas about how to mitigate the overconfidence.
Grant: That's very helpful. It actually connects to one of your
other really interesting points in the book, which is about using better
statistics. Can you talk to us a little bit about how you would do that as a
business leader?
Mauboussin: We're awash in statistics. You watch a ball game or you
read the business page of a newspaper, and we know that they're not all created
equally. So what makes for a useful statistic? What I basically argue for -- I
got this from the sabermetrics guys, the sports statistics guys -- is that you
really want two things. One is that you want persistence and the second is that
you want predictive value. Persistence simply means that the actual statistic
is correlated from one period to the next. For instance, if I know your batting
average for 2012, it would correlate highly with your batting average in 2013,
or how well it would correlate would be a measure of persistence.
The second thing is predictive
value. You want that statistic to actually correlate highly with the objective
you're trying to achieve. For example, in baseball, on offense you're trying to
generate runs. The question is how well does batting average correlate with run
production? Let me tie this back to [the film] Moneyball. There are a
lot of different themes in Moneyball, but one of them was a simple one,
which is that one base percentage is a better measure of performance than
batting average. What they found was on-base percentage has a higher
correlation from one season to the next, which means it's more indicative of
skill than batting average, so it passes the persistence test. Secondly,
on-base percentage actually correlates higher with run production than does
batting average. That's going to say that is a superior statistic because it's
more persistent and it's more predictive. I should have backed up and said that
high persistence is almost always indicative of high skill, and low persistence
typically [is indicative of] lots of luck.
Grant: Let's take an exception to that rule, which is when you
think about the interdependence of different players on a baseball team, or for
example in a company, too, you could assume that on-base percentage is
attributable to skill, but then you end up batting right after another highly talented
batter, and that's going to increase in general your on-base percentage, right?
How do you decouple the individual scale from the context in which you find
yourself?
Mauboussin: Super difficult, right? When I did the work on on-base
percentage, I actually did try to size up the problem you just articulated. I
did it on a team level versus an individual level. Of course, they're going to
roll up on some level, but you're right. It's a major step in the right
direction, but in some ways it can be a fairly blunt instrument. The really
careful statistical people try to understand exactly if you're batting in a
different order what impact will that have, and they try to measure that out
and extract that effect.
But this also leads to another point
that's broader.... For example, take sports that are very simple. Tennis is one
on one. We have a large sample -- if we play a five set match, there's a large
sample size, right? So we pretty much know by the end of the tennis match which
of us has been more skillful. But you get into a football game, there are a lot
of players, a lot of interaction, and identifying those effects of individuals
is a vastly more challenging task. Again, it doesn't mean you shouldn't try, or
try to gain some insight into doing that. But, you're right, the degree of
difficulty goes up as you add complexity -- and of course, with corporations,
it's the same thing. There's a lot of moving parts; it's very difficult often
to say that this, that or the other caused one thing or another.
Grant: You mentioned Danny Kahneman's work earlier. He's added
another really interesting variable to the equation, which is, are you working
in an environment that's stable and predictable or much more turbulent? He's
made the point that you can rely on your skill and your expertise and your
intuition much more in a predictable environment. But most of us don't have the
luxury anymore of working in these very predictable environments. How do you
think about navigating a more uncertain world?
Mauboussin: This is such a fascinating topic. I wrote a chapter about
this in my prior book and I called it the "expert squeeze." I
basically said the main way to think about expertise is to think about
precisely that continuum you just laid out. In some fields, the environment's
stable. I like to say it's stable and linear, so cause and effect are very
clear. In those realms, you can train your subconscious to be really good. The
challenge is that increasingly, especially in a business setting, computers are
taking on those tasks, professional tasks, and can do them very efficiently and
cheaply. Experts are good at those [tasks], but increasingly there's an
encroachment from technology. That's a challenging situation.
The opposite extremes you point out
are environments that are unstable and non-linear. There, we know that experts
are very poor predictors. There's really no way to train your system one, your
subconscious. I always love to make this distinction -- especially since I'm in
the finance business -- the big deal between experience and expertise. People
often think that experience and expertise equal each other, and that's true on
the stable side of the continuum. But when you're on the unstable, non-linear
side of the continuum, you don't really have a predictive model that works. The
key to expertise is having a predictive model that works. There we know that
experts do very, very poorly in their predictions. By the way, this is early
2013, this is the time of year everybody's making predictions about what's going
to happen, and of course, most people who keep track of those things know that
they're notoriously very poor.
What's interesting, though, is what
we're seeing in some cases called the wisdom of crowds, that collectives can be
more effective than experts in making judgments in those kinds of areas under
certain conditions. I call this the expert squeeze, because experts are getting
squeezed on this unstable, non-linear side by wisdom of crowds properly
harnessed. They're getting squeezed on the other side by computers and
technology. There's less space in the middle for the experts to navigate than
they used to have. This is the fundamental first question to ask: Where is the
problem I'm trying to think about on that continuum from stable [and] linear to
unstable, non-linear? That really dictates a lot about how you should think
about solving it and by what means and techniques as well.
Grant: To build on that, it seems like unlearning is a big part of
the equation there. There's some work that Nancy Rothbard here at Wharton was
involved in showing that when people move from one company to another, they end
up getting hurt by experience because they carry a lot of baggage with them
about what worked in a particular context that's no longer relevant to their new
context. Do you have any wisdom to share about how to unlearn some of those
things?
Mauboussin: No, I don't, but obviously this is a big theme in all of
social psychology, which is the context of where you are is incredibly
important in shaping the decisions that you make. You have certain experiences;
you're socialized within an organization a certain way. Those experiences and
even imprintings ... deeply carry you through your decision-making for often
the rest of your career. It's a really hard thing to unlearn. But it can be at
the same time very useful just to be mindful that while we'd love to think of
ourselves as rational and objective and fact-based in our decisions, social
context -- be it new or old organization or whatever's going around you -- is
deeply influential in how you decide. That inserts a lot of humility but maybe
raises awareness to help people get more effective at making their decisions.
Grant: You work in the world of finance. How do you take all this
knowledge and apply it to achieve success in your own job?
Mauboussin: There are a number of different angles on that. One is what
I like to call macro-aware but macro-agnostic, which is to say spend as little
time as possible predicting big things in the world. You have to be aware of
what's going on obviously and how those things may in fact have an impact on
various scenarios that might happen for a company or an economy. But try to be
macro-aware, macro-agnostic. The second thing is just thinking about what
statistics are useful. For example, which money manager is likely to succeed in
the future? We know that past results are typically an ineffective way to
anticipate future results. But an isolation on process -- a manager's process
-- might give us a better insight. There are statistical ways that we can start
to get a glimpse at process that can be very helpful.
[Also,] you mentioned overconfidence
before.... That's also rife obviously in the investment business. Even as an
analyst trying to anticipate a company's performance, one of the classic ways
that that shows up is people project ranges of outcomes, for example, sales
growth rates or profit levels, that are vastly too narrow. In other words,
they're overconfident in their own ability to understand the future, so just getting
people to widen out those ranges, to think more robustly about that can be
very, very helpful. There's almost no facet of finance where these ideas don't
touch and can't help. Very hard to do, but awareness and tools and techniques
to try to manage it, to minimize the mistakes, is of great value.
Grant: You manage to do all of this and keep up to date on the
latest evidence that might inform these practices. How do you juggle these two
things simultaneously?
Mauboussin: Part of it is that I'm pretty bad at everything. A lot of
it is just a natural curiosity. Also one of the things that's been very helpful
for me is teaching as an adjunct, so I'm not a real professor like you are -- a
heralded professor -- but being an adjunct for me has been very helpful. In
part, the way I think about it is to try to take the very best of what
academics bring to the table and the very best of [what practitioners bring].
What academics do that's very helpful is tend to be rigorous, using the
scientific method to understand and explore ideas. But they're not always
totally practical. What the practitioners bring to it is [to say], "Hey,
we have to make money and we have to have a practical angle on it." Taking
the best of both of those worlds and trying to combine them is what's been for
me the most satisfying aspect of this.
Again if there's something you can
draw from the world of academia that can improve your performance in some way,
that's great. I'm about to start my 21st year teaching at Columbia.
When I started there, there was really no behavioral finance program. In fact,
I often recommend the students take negotiation courses, because that was the
closest you got to sort of tapping into some of these ideas from what we now
call behavioral finance. That's obviously come a long way. These are
extraordinarily useful ideas, but shockingly there are whole generations,
including my own generation, of people who never learned this in the classroom.
Unless you go out on your own, in effect, and learn these things now and try to
put them into your process, you have a blind spot in a lot of your decision
making. That's also a fascinating thing. A lot of people running corporations
have never learned about these ideas, and they have this blind spot. Trying to
fill that in a little bit has been a really fun activity.
http://knowledge.wharton.upenn.edu/article.cfm?articleid=3204
No comments:
Post a Comment