MANAGEMENT SPECIAL..... Effective Teams and Managers: What Google Has Learned

Effective Teams and Managers: What Google Has Learned

As the director of People Analytics at Google for the last decade, Brian Welle’s world revolves around data. He has found that those hard, cold numbers can, when used properly, uncover the key attributes that make people better managers and team members. Once identified, the attributes can then be cultivated and instilled to boost performance. Welle spoke recently with Cade Massey, Wharton practice professor of operations, information and decisions and co-director of the Wharton People Analytics Initiative, about applying data analytics in the office. Massey is co-host of the Wharton Moneyball show on Wharton Business Radio on SiriusXM channel 111, and this interview was part of a special broadcast on SiriusXM for the Wharton People Analytics Conference.

Welle identified eight attributes that define a top-performing manager: a good coach; empowers the team and does not micromanage; expresses interest in and concern for team members’ success and personal well-being; productive and results-oriented; good communicator — listens and shares information; helps with career development; has a clear vision and strategy for the team; and possesses key technical skills that help him or her advise the team.

An edited transcript of the conversation follows.

Cade Massey: You have a Ph.D. in industrial and organizational psychology from New York University and were a postdoctoral fellow at Harvard’s Kennedy School of Government. How did you end up at Google?

Brian Welle: I did the post-doc because I thought I might want to be a professor. Then I saw how hard professors actually have to work, and I decided maybe that wasn’t for me. This role I applied for in 2006 could not be described to me by my interviewer. He basically said, “We’ve got a lot of data and a lot of challenges on the people side, and we need smart people who know how to do research to help us figure it out.” That was the lack of structure I was looking for at the time.

Massey: You told a story about a scene from Star Trek, which you felt perfectly captured what you wanted to do in life. I would love to hear that story again.

Welle: I honestly believe that I am an IO psychologist today because I watched Star Trek: The Next Generation when I was in high school. There was one character that really stood out to me across the whole show. Oddly, when I ask people about Star Trek: The Next Generation now, that is the one character they never really think about. But it was Deanna Troi, and she was the ship’s counselor. She was the one who negotiated cross-cultural agreements, she was the advisor to the captain, she did the crew rosters, and she was the only person on the bridge that did not have to wear a regulation uniform. I thought, how could I be the ship’s counselor to an organization? I sort of feel like that is what my role is at Google.

Massey:  How would you describe what you do at Google, and where are the connections between that and the Star Trek character?

Welle: Deanna Troi really had to understand why people were behaving the way that they did, and she had to sort of impose a logical explanation for what was going on so she could make those interactions better and save the crew and all sorts of other fun things. The amazing thing about organizations is, as human beings we all know how to do the basics. We know how to interact with each other. We can get through the day. We can work in teams relatively productively, but there is so much room to optimize all of that. If you can take a step back and understand the dynamics, you can actually help the system work better. You can give people the tools they need to self-regulate. You can put rules in place when you need to. There are so many opportunities for efficiency that a lot of organizations don’t even know they need, because they’re people who say, “I know how to do all of this stuff because I’ve been doing it my entire life.”

Massey: I know that Google is not a very rules-happy place. In fact, they’re anti-rules. Can you give us an example where you realized you needed to put a rule in place?

Welle: I think a good example is our approach to management. Hierarchy is like a necessary evil in most organizations. Of course, we had managers at Google, but we did not provide them with a lot of structure. We said, “We want your teams to be productive. We want you to do good work. We will not stifle you with rules on how to get that done.”

We were getting some signals that managers on the whole were not performing or at least not providing the experience to the people on their teams you would want to have when you come to work. We did research on what differentiates the best managers from the not-so-great managers and came up with eight attributes of success — everything ranging from having consideration for people as people to providing coaching and career development advice. We codified that set of eight, and we give every Google employee the chance to rate their manager on those eight attributes, and we provide them with the feedback. That is an example of a set of eight rules that is empirically based. We know it drives good outcomes, and if you can prove that a set of rules is leading to better outcomes, people will listen to that. It just needs to be based on data and evidence.

Massey: You’re very well known outside of Google for unconscious biases and work on diversity. You’ve been working on judgment biases within Google for probably 10 years. I feel like teams are a place where analytics hasn’t made much progress, so can we hear more about this teams research?

Welle: I think that there are two really big, important topics within organizations that are incredibly difficult to study. One is leadership and the other is teams.

Part of the complexity is that we all feel as human beings, we work in teams, and we look to leaders, so we feel like we intuitively understand it. Yet when you try to actually study it, you realize they’re messy. The rules are difficult to determine, so we leave a lot of it unstructured. We didn’t want to let teamwork just happen at Google because we thought there might be an opportunity to help teams be more effective and to create a better team experience.

There are a lot of requests coming from engineers who are doing work. Our engineering population tends to be very skeptical about the status quo anyway, so we’re asking, “What do we know? You have so much data about us. What can you tell us about what will help us work better together?” That was the genesis of the work. What we were expecting to find is that we would have a long list of individual characteristics that would help us determine the composition of an effective team.

You would want a 10-year mix. You want to make sure you have gender diversity, you want extroverts, you want introverts, you want highly conscientious people — you want that whole mix. If we could quantify that mix, we could come up with the ideal recipe for an effective team. What we found instead was that those individual factors did not predict team success. What predicted team success is what the team members themselves were actually doing in their interactions with each other. What kind of environment were they creating? How much could they depend on each other? How clear were their goals?

You can absolutely measure those things. Psychologists have been measuring them but didn’t necessarily put them together in one study. Now that we know that, we can distribute those measures to teams so they can take a self-diagnostic and at least have the language and the structure to make improvements to their team process.

Massey: What does this look like in action?

Welle: We created diagnostics. We tested it to make sure that it was valid, internally consistent, had a good user experience. We created an online survey. If you are a member of a team, you can input the names and the email addresses of all of your team members into the system. If everyone agrees that they are ready for this experience, the surveys get sent out, people fill it out and it gets aggregated. Each team gets a report on each of these dynamics that we found to be important, and that’s the starting place for the conversation.

Massey: How do teams react to that? I would think in most organizations, people are sensitive about giving critical feedback in that kind of setting.

Welle: It’s usually perceived very well. First of all, it is instigated by the team, so they want the feedback. Usually what the diagnostic does is gives you information you sort of felt already about the team, but it helps you channel that. If the one place where your team scores extremely low is on psychological safety, then providing you with that feedback can say, “It says psychologically unsafe. You people figure it out.” We usually need a trained facilitator to come in and broker that particular conversation.

Massey: One last observation I want to hear is a very practical tip for those working remotely. I hear you and Elizabeth McCune at Microsoft talk about what it means to work away from headquarters. You say you have a recipe for what makes this work. What’s your recipe?

Welle: I worked in headquarters for two years before I left, which allowed me to create really good, robust social network. You have to have the skill to collaborate in meetings over video conference, and that’s something you can learn over time. What’s great about our team’s research is one of the things I was afraid that we would find is that the distributed nature of team would negatively impact effectiveness, and they would force me to move back to headquarters. Fortunately, no correlation.

Massey: What are working on that what can we anticipate hearing about from you two years from now?

Welle: You might be tired of us hearing about unconscious bias and the diversity narrative, but we started this journey a few years ago with a concerted investment in unconscious bias training and other diversity-related activities. It has just skyrocketed since then. We have made some public declarations about diversity and where we want the tech industry to go. I sincerely hope that is what you’re going to hear us talk more about in the next six months, year, two years.

http://knowledge.wharton.upenn.edu/article/brian-welle-wharton-people-analytic-conference/?utm_source=kw_newsletter&utm_medium=email&utm_campaign=2016-05-26

BOOK SPECIAL...... ‘Negotiating the Nonnegotiable’ at Work and at Home

BOOK - ‘Negotiating the Nonnegotiable’ at Work and at Home

Daniel Shapiro, director and founder of the Harvard International Negotiation Program, has negotiated some of the most challenging conflicts with heads of state, corporate executives and even families. Shapiro recently joined us on the Knowledge@Wharton show on Wharton Business Radio on SiriusXM channel 111 to talk about his new book on the subject, Negotiating the Nonnegotiable: How to Resolve Your Most Emotionally Charged Conflicts.

An edited transcript of the conversation follows.

Knowledge@Wharton: I found it interesting that your book takes a look at both ends of the spectrum, both the personal and the professional. When it comes to resolving conflict in the office, a lot of the same types of incidents and issues arise at home and work that could be negotiated in the same kind of manner.

Shapiro: Yes, our research has shown that the underlying dynamics that make our conflicts so miserable — whether at work with a tough colleague or at home with a tough spouse — tend to be quite similar. The power of them might be more at work or at home, but the impact is just the same. It’s difficult.

Knowledge@Wharton: You break it down into a very simple and straightforward point: Whether you’re talking about conflict in the office or conflict at home, the biggest common denominator is that both involve humans. And humans are still a very flawed species.

Shapiro: We are both a flawed species and an incredible species. How do you deal with these emotionally charged conflicts — with your board, with somebody at work, at home? On the one hand, fighting back doesn’t work, it just escalates the conflict. Two, ignoring the conflict doesn’t work because the conflict continues to fester. Three, and here’s the twisted part, even if you try to collaboratively work things out — with your spouse, with your tough teenager, at work — in emotionally charged conflicts, even that tends not to work. Because we’re not getting to the underlying dynamics that are at play, the underlying emotional forces that tend to drive us toward conflict.

Knowledge@Wharton: What are those forces? What are people just missing that would enable them to resolve difficult conflicts?

Shapiro: Let me start with at least one example. In the book, I talk about a concept that my research has shown is quite powerful in any conflict, and I call it vertigo. Think about the most recent conflict that you’ve gotten into. You don’t need to tell me what it is — at home, at work. Vertigo is when you get so emotionally consumed in that conflict that you can think of nothing else other than that evil person who did not consult you before making that decision, or whatever the problem is. You go home after that long day of work, and yes, you are physically at home — your body is at home — but your mind is still racing about what happened at work. You are in vertigo. We all know the experience, but to give it a name empowers you to decide, do I want to go there, toward vertigo? Or do I want to try to have a collaborative, positive conversation here?

As an example, just last week I was working with Israelis and Palestinians in the Middle East, and sharing with them some of the ideas of this new book, some of the new research, including this concept of vertigo. I got an email two days ago from one of the very senior players on one of the sides of the conflict. She said to me in that email, “I just got out of a meeting right now.” This was a meeting between sides, in fact. She said, “As I was there, in that meeting, I felt that tornado of vertigo moving toward me,” ready to sweep her away. She said, “I thought in my head, do I want to go there? Do I not?” She decided no, and she said it was an incredibly productive conversation. We all have that choice, whether it’s vertigo or some of these other research points in the book.

Knowledge@Wharton: In some respects, taking a different approach ends up being the best path then, correct?

Shapiro: Absolutely. But it’s hard to do. Another concept that we’ve really worked to mine in the book is a concept that Sigmund Freud initially called the repetition compulsion. This is the idea that we tend to repeat the same dysfunctional patterns of behavior again and again and again and again, even though we’re shooting ourselves in the foot. You can send all of your employees to corporate training on negotiation. They come back from the training, and they think, “Oh, I’m much better.” Yes, they might be better for a day or two, but unless you deal with these underlying dynamics, your solidified and imprinted patterns of conflict resolution, unless you can deal with these and try and really work at them and change them, it’s lost money for your company.

Knowledge@Wharton: You talk also in the book about the taboos that exist in the office, and can be in the family as well.

Shapiro: Yes, whether it’s the office or home, there are those issues that are socially prohibited from talking about. Yet it’s often those issues that are the ones that are driving so much of the dysfunction either at work or at home. In the home setting, everybody knows it, but nobody can talk about mom’s drinking problem. Or at work, everybody knows that there is dysfunction going on in that one department, but nobody dares tell the CEO or the senior executive for fear of getting socially punished in some way. Yet if you don’t talk about those issues, you’re suffering. It affects the bottom line.

Knowledge@Wharton: You did an interesting experiment, which I want you to share because it was a rather unique tack in terms of trying to mitigate and work through what you call the Tribes Effect.

Shapiro: Yes, most senior business people think, “Ah, I’m immune to that stuff. I’m good at conflict resolution.” As you said at the beginning of our conversation, people are people; we are all human beings. There’s this exercise I did at Davos at the World Economic Forum. The first time I did it there was back in 2006. Forty-five global leaders come into the room. I divide them up into six groups. I ask them to create their own tribes at their tables — what are their values, what are their beliefs, so on. I ask them to dress up in their own tribal outfits. Literally, I have the greatest [stories for] blackmail in the world — a deputy head of state in that room with balloons on his head, VCs helping to create some of the most well known companies, and so on.

They create these six tribes, and then I get up in front of the room and say, let’s debrief the exercise. All of a sudden, the lights go completely black in this room, and into the room bursts this intergalactic alien who says, “I am an intergalactic alien. I have come to destroy Earth. I will give you one opportunity to save this world from destruction. You must choose one of these six tribes to be the tribe of everybody. You cannot change anything about your own tribe, and if you cannot come to agreement by three rounds of negotiation, the world will be destroyed. Ha! Ha! Ha!” And out floats the alien.

Three rounds of negotiation, the intensity builds and builds. They are talking rationally, emotions start to pick up, and by round three, in the middle of the room, you have six chairs. Five men and one woman come to the middle of the room. One of the most well-known VCs in the world is one of those negotiators, [as well as] a media mogul, a president of a university. These men start yelling over one another.

They start yelling over this woman, “Our tribe!” “No, our tribe!” “No, ours!” This woman gets so upset that she literally stands on her barstool, and she yells, “This is just another example of male competitor behavior! You all come to my tribe!” One other tribe comes to hers. The others refuse. Five, four, three, two, one, boom! Our world explodes at Davos. I have run this exercise dozens and dozens of times with groups around the world, and almost always the world explodes again and again.

Knowledge@Wharton: What’s the reaction after the fact, by all these executives?

Shapiro: This is our reality right now, and this is a beautiful question because it’s exactly the point of the book. Someone who is listening to your show right now might be in the midst of a miserable divorce, and yet on some rational level, you’re not supposed to be mean to your spouse that you’re divorcing. You have two kids or whatever it is.

At work, I should not be doing this behavior with my colleague. You get sucked into it. That’s the problem. The book offers ideas on how to deal with it. How do people deal with it? At Davos, I asked the group afterward, how do you feel? This one guy points to me, and he says, “This is all your fault!” He said, “You set us up for this!” I said, “You know what? I did everything in my power to try and make the world explode, but at the end of the day, you had a choice. You could have saved the world.” That is the reality.

Conflict is one of the greatest costs on any business, any company, in any family, and it is also one of the things that we have more than any other power to do something about. Exactly to your point at the beginning because we are human beings, and this is a human problem, there is a human answer.

Knowledge@Wharton: But a lot of the time the conflict, and you talked about this as well, is something that you can’t see with the naked eye. It’s just behind the scenes and festering a lot of times.

Shapiro: Yes, and at the same time, if you take just a little step back, there often are a small set of factors that tend to drive a lot of our conflicts. Let me lay out a few that I talk about. One, autonomy: this core motivation to have the freedom to make decisions without somebody else imposing a decision on you. At work, if one of your teammates makes even a small decision but didn’t consult you, it can have a big emotional impact. That’s autonomy — recognize it.

Let me give you just one more. Status: We all like to feel high and good in our social standing. If someone comes along and says — “Yeah, why did you do that?” — you suddenly feel smaller, shrunken. Those are the kinds of things we can be aware of. What are the underlying emotional factors that tend to stimulate our negative emotions and get us all revved up?

Knowledge@Wharton: How much harder is it for people who are not only dealing with this type of stress and conflict at work, but then they have it at home? Being the dad of a 9-year-old and two 7-year-olds, I get this quite a bit right now.

Shapiro: So you have a 9-year-old and two 7-year-olds. I have a 10-, an 8-, and a 4-year-old. So we’re quite similar in our family lives. It’s the same tools. Are the tools perfect? No. This negotiating the nonnegotiable, is it a quick fix? If you read the book, you’re going to have beautiful relations immediately? No, and that’s not the point. The point is it takes work, but there is a path to get there.

At home, for example, my wife and I, we both know this concept of vertigo, and she doesn’t reject it even though I’ve been working on it. She’ll be the first to say to me, “Hold on, Dan. We are moving toward vertigo. Do you really want to go there now?” Some of the time, it’s like, “You know what? I do.” Some of the times, no, I don’t. But we’re taking back power: We’re not letting the conflict control us; we’re controlling the conflict.

One other connected point: A huge point is to shift the dynamic, so it’s no longer me versus that other person in the conflict, it’s the two of us facing the same shared conflict. It’s not me versus my colleague, me versus my wife. They are not the problem. The problem is the conflict. How do we deal with it? That mindset shift is what takes people out of that Tribes Effect, that adversarial mindset, and what allows people to save the world.

Knowledge@Wharton: You talk about how identity is such a strong factor in this. There are many times when people hold things sacred, which is the term you use. How do you deal with that?

Shapiro: Business people might think, “Wait a minute, sacred! That’s outside the workplace, that’s the religious element.” My notion of the sacred is anything that you find deeply meaningful, within your organization or beyond. It can be religion but it is much bigger — it’s sacred.

Looking at the current situation between Apple and the government, the FBI, around the phone situation. There is a sacred value that it appears that Apple holds, which is privacy of information. They are willing to sacrifice a lot to risk elements of their reputation for that. So the sacred is there with us. One huge point is, just recognize it. And even more than that, think carefully — with your various different teams, with the leaders of teams in an organization, what do you hold sacred? What is your identity within the organization?

In the book, I talk about a simple model called BRAVE, you can walk and think through it with your organization. What are the beliefs that are important to your organization or to you personally? What rituals do you find are useful to do — the holidays, the monthly outing that your team does? What are the allegiances that are important to your company? What are the values that are most essential? What are those emotionally meaningful experiences that define your organization? Way back in the day, when all those Microsoft, Apple people were in a garage building their computers, that was a big deal, and that story still affects these companies to this day.

Knowledge@Wharton: Do the companies themselves truly understand how important a topic this is — not only to the bottom line but for the corporate culture as well?

Shapiro: Your point hits it dead on. When a company looks at their financial spreadsheet, what they see is, one, cost, typically around conflict, which is the legal fees, the litigation and so on. Yet, like you are saying, the burden of conflict is tremendous with all of the hidden costs. You have the poor decision-making because that team is so dysfunctional. … It is extraordinary, in terms of cost, the amount of time devoured when resolving conflicts between the disputants [and people who get involved such as] the manager, HR, the [folks doing the] grievance investigation.

You have your superstar players that end up walking from your company to go next door to your competition. Is it about the salary? Absolutely not, it’s because I can’t stand you guys. Then you get the toxic corporate culture, the damage to the reputation, and on and on and on. And it stacks up to be exponentially more.

Knowledge@Wharton: If you get to the point where you really just have irreconcilable differences, how do you deal with that?

Shapiro: That’s the number one reason I get called into organizations to consult. A CEO brings me in, closes the door to the office and says to me, “Look, how do you deal with irrational people?” The moment I hear that question, 99% of the time, a siren goes off in my head and says, you know what? This is completely negotiable. What people tend to do, once you get into this adversarial mindset, all of a sudden I am closed off to the other side, to their world. I start to believe my way is right and legitimate; their way is wrong and illegitimate. I am living in a closed world.

So often what I’ll do with people in the corporate world, and even in the family world, in crisis, is to have them literally move seats. I want you sit over here in this other seat right here, take on the role of that other person who they find completely irrational. We systematically walk through what things look like from your perspective as the other party. This little exercise alone has allowed breakthroughs in international conflicts — Peru and Ecuador is an example.

I was working with someone recently who was doing a multibillion dollar merger, and it hit some problems. It was a problem between the two CEOs; they just didn’t like each other. I did it there, and it broke the impasse. These aren’t foolproof methods. I’m not saying these are all the answers, but this stuff does work and it can save extraordinary amounts of money. It leverages the organization. It helps your family.

http://knowledge.wharton.upenn.edu/article/negotiating-the-nonnegotiable/

MANAGEMENT SPECIAL .........Daniel Kahneman’s Strategy for How Your Firm Can Think Smarter

Daniel Kahneman’s Strategy for How Your Firm Can Think Smarter

Nobel economics laureate and psychologist Daniel Kahneman — considered the father of behavioral economics – retired from his teaching position at Princeton a few years ago to co-found a consulting firm in New York. In a talk at the recent Wharton People Analytics Conference, he said of his consulting experience that he had “expected to be awed” by the quality of the decision-making in organizations “that need to make profits to survive in a competitive world.”

“I have not been awed,” he stated.

“You look at large organizations that are supposed to be optimal, rational. And the amount of folly in the way these places are run, the stupid procedures that they have, the really, really poor thinking you see all around you, is actually fairly troubling,” he said, noting that there is much that could be improved.

Figuring out how to make the act of decision-making “commensurate with the complexity and importance of the stakes” is a huge problem, in Kahneman’s view, to which the business world does not devote much thought. At the conference he described how significant progress can be made in making organizations “more intelligent.”

The Problem with People

If individuals routinely make poor decisions as Kahneman says, why is that the case? The answer lies in behavioral economics, a field which explains why people often make irrational financial choices and don’t always behave the way standard economic models predict. (Kahneman explained much of his work in his much-lauded 2011 international bestseller Thinking, Fast and Slow.)

Behavioral economists believe that human beings are unknowingly hamstrung by overconfidence, limited attention, cognitive biases and other psychological factors which inevitably cause errors in judgment. These factors affect everything from how we invest in stocks, to how we respond to marketing offers, to how we choose which sandwich to buy for lunch.

“We’re fundamentally over-confident in the sense that we jump to conclusions — and to complete, coherent stories — to create interpretations,” said Kahneman. “So we misunderstand situations, spontaneously and automatically. And that’s very difficult to control.” Furthermore, he said, much of human error is not even attributable to a systematic cause, but to “noise.” “When people think about error, we tend to think about biases…. But in fact, a lot of the errors that people make is simply noise, in the sense that it’s random, unpredictable, it cannot be explained.”

He cited some disturbing evidence about the professional judgment of experts: “You put the same X-ray in front of radiologists, and about 20% of the time in some experiments they don’t reach the same diagnosis.”

From his consulting work, Kahneman offered an example from a large financial institution in which loan approvals and insurance company judgments are routinely made. The decisions, involving hundreds of thousands of dollars, frequently hinge on the opinion of a single individual. Kahneman mounted an experiment in which team leaders were asked what percentage they thought two different professionals’ decisions would vary if they were each asked to evaluate the same case.

 “Many people give the same [guess]: somewhere between 5% and 10%,” said Kahneman. “But the answer is between 40% and 60%. It’s an order of magnitude more. It’s completely different from what everybody expects.” He noted that at the organization in question, there was “a huge noise problem” of which the leaders were completely unaware.

The problem cannot be chalked up to the relative inexperience of some employees, according to Kahneman: “What was very surprising, at least in our experiments, is that experienced professionals were as variable as novices.”

Could it help matters to have experts arrive at decisions together, as a group? Even if this were feasible in organizations, which it is often not, there are pitfalls here as well. Kahneman said that according to social psychology, when a group of people discusses a case there are “huge conformity pressures” that lead participants to radically underestimate the amount of disagreement among them.

In the face of what seem like daunting odds that business decision-making can be improved, what’s a company to do?

The Cure: Algorithms

Kahneman’s prescription is for organizations to temper human judgment with “disciplined thinking” through the use of algorithms. The indications from the research are unequivocal, he said: When it comes to decision-making, algorithms are superior to people. “Algorithms are noise-free. People are not,” he said. “When you put some data in front of an algorithm, you will always get the same response at the other end.”

A good algorithm does not require a massive amount of data, said Kahneman. (He said this was “a secret not widely known in the financial industry.”) Let’s say you are evaluating the financial stability of firms, he said, for example to give them a loan or insure them against financial risk. His recommendation is to sit down with a committee of people who are knowledgeable about the situation and make a list of five or six dimensions. More than eight is probably unnecessary. “If you create good ranking scales on those dimensions, and give them equal weight, you will typically do just as well as with a very sophisticated statistical algorithm.” And do just as well — typically much better — than experts on the average, he added.

Kahneman, who is Israeli-American and now in his 80s, recalled inventing a prototype of this procedure when, as a young Israeli platoon commander with a psychology degree, he was asked to set up a new interviewing system for the army. Though met with some resistance at the time, he said, the system he designed is actually still in use by the Israeli armed forces.

Kahneman identified six dimensions that could be rated one at a time, among them punctuality, sociability, conscientiousness, “something called masculine pride” (he noted that these were interviews for combat units, 60 years ago), and others. “Very important to rate things one at a time,” he observed. “That way you don’t form a global impression of the person, but a differentiated impression of [each] topic. It controls what psychologists call the halo effect.”

The entire selection process was to consist of generating the six scores and adding them up. When many interviewers complained — one saying “you’re turning us into robots” — Kahneman said he added a final “global rating” step as a concession to human intuition: “Then, close your eyes and think what kind of a soldier this person is going to be. Put down a rating between 1 and 5.”

When the new interviewing system was validated against actual performance a few months later, said Kahneman, it turned out that the final global rating was very accurate. In fact, it was much more accurate than any of the single dimensional ratings. “But there is a lesson to be learned,” he stated.  Previously, candidates were interviewed with only a global rating, and “it was worthless.”

“Global rating is very good — and intuition is very good — provided that you have [first] gone through the exercise of systematically and independently evaluating, the constituents of the problem,” he explained. “Then when you close your eyes and generate an intuitive, comprehensive image of the case, you will actually add information.”

Implementing this type of procedure in any organization can of course meet with resistance from employees, said Kahneman. “You have to do it with a light touch, because otherwise people will hate you and not comply. But if you do it in a way that they view as helping them perform their tasks, it’s not too bad.” In his experience, he said, if you put effort into guiding people to look at information in a particular way, they actually find that it helps them do a good job.

How do company leaders respond when they are first told they should implement algorithms to guide their experts’ opinions? “Not very well,” he said. But “when you tell team leaders that there is 50% variability when they expected 5% or 10%, then they’re willing to take an algorithm.”

Will Artificial Intelligence Replace Human Intuition?

Kahneman was asked about the growing role of artificial intelligence (AI) in business thinking: In particular, about powerful algorithms that are increasingly performing tasks previously done by accountants, consultants and managers. His response surprised many in the audience: “I think I’m quite worried about it.”

His concern is that as AI becomes more sophisticated, it is moving beyond simply helping humans achieve disciplined thinking to actually being able to execute professional judgment on its own. This will be “very threatening to the leaders of organizations,” he said. “Because once you have decision analysis, anybody can outguess the leader…. How will this affect the power structure?”

He cited as an eye-opening AI milestone the fact that earlier this year, a Google computer program beat the world champion of the popular Asian game Go — who had won 18 international titles — four games out of five.

“Go is supposed to be the example of … an intuitive game…. The real experts cannot explain exactly how they reach their conclusions: it’s too complicated.” But, said Kahneman, the Google team had built software based on 150,000 actual human matches, and the software then improved by playing the game against itself about 30 million times. That, he said, is how you end up with a program that is superior to the world champion: the program had access to more information than one human being possibly could.

“All of this depends on the availability of data: this is how intuition develops,” he noted. “We develop intuition with the data we collect in a lifetime. AI will be able to do better. How will we live with that?”

http://knowledge.wharton.upenn.edu/article/nobel-winner-daniel-kahnemans-strategy-firm-can-think-smarter/?utm_source=kw_newsletter&utm_medium=email&utm_campaign=2016-06-09

HR SPECIAL..... An Imperfect Test: The Problem with Job Performance Appraisals

An Imperfect Test: The Problem with Job Performance Appraisals

Wharton management professor Peter Cappelli has spent decades studying the complicated dynamics of employment. In a post-recession world, his research is more timely than ever as companies large and small struggle to adapt to a new normal that relies on fewer employees handling a larger, shifting workload. One practice that has persisted in this changing business landscape is the ubiquitous performance evaluation, which Cappelli describes as universally despised by both supervisors and subordinates.

In their latest research, Cappelli and colleague Martin Conyon, a professor at Bentley University in Massachusetts and a senior fellow at Wharton, question the usefulness and accuracy of performance appraisals and find some surprising answers. Cappelli, who is also director of Wharton’s Center for Human Resources, discussed their findings in a recent segment on the Knowledge@Wharton show on Wharton Business Radio on SiriusXM.

An edited transcript of the conversation appears below.

Knowledge@Wharton: Is the performance appraisal as important now as it was 20 years ago?

Peter Cappelli: It’s more important in the sense that more people have to do it. If you look around the United States, there haven’t been a lot of recent studies. But the last ones that were done show that more than 90% of the work force has a performance appraisal. Basically, if you’re not in a union — where they tend not to get them because of collective bargaining agreements — you’ve got a performance appraisal. The federal government mandates it for employees. State governments do it. The Army does it. The Navy does it. I think the big change has been when you leave the United States. It used to be kind of a U.S. thing, but now you see them all around the world.

Knowledge@Wharton: Are companies seeing a level of importance to performance appraisals that they want to bring to their business strategy?

Cappelli: I think some of it is just the increasing attention to the workforce and recognizing that managing your employees is a smart thing to do, given how important they are to the organization. You’re crazy if you don’t, right? And a lot of companies outside the U.S. just copy what they think are best practices in the U.S., even if they’re not sure why they’re doing it. I think that’s why it’s spreading around the world. You see it all through India and China. Unless you get to the countries like in the Middle East, which still have Soviet-era labor and employment laws — you probably don’t see them very often there. But otherwise, all over.

Knowledge@Wharton: The work with Martin Conyon and this paper is trying to bring together information from a variety of sources about performance appraisals?

Cappelli: The thing about performance appraisals is they are ubiquitous. There’s probably nothing in the field of management that is more common. And there’s also almost no practice in the world of business that people hate more. The evidence on this is pretty overwhelming. It’s also surprising how little we actually know about it. There’s an awful lot that’s been done on psychologists with little slices of the performance-appraisal question. Mainly, what psychologists are interested in is, how did the person [doing the] rating and the person being rated get along? And how do the characteristics of the rater and the ratee affect the results? One of the things that we know from this is one of the best predictors of your score is bias. That is, how you and your appraiser map onto each other. Are you similar? [Then] you get higher scores. The more different you are in terms of ethnicity or age or sex, the less well you’re going to do.

That’s one of the things we know. But how do they actually work inside companies? Quite remarkably, almost nobody has looked at this. We got data from a large Fortune 50 company on all their performance appraisals over a 10-year period. There were a couple of the questions that we were after. One of them is, there’s a kind of view in a lot of places and among a lot of executives that employment is like a contract. At the beginning of the year, you set goals, then we assess how well you’ve done. At the end of the year, we give you a pay increase based on how much you have achieved of your goals and how well you’ve done, maybe compared to everybody else. But there’s another view that it is not like a contract. That it’s really kind of a relationship. If you think about employment, you don’t really have a contract with your boss. The boss is telling you to do different things all the time. And based on what they’re hearing from their bosses, [employees] decide, “Oh, I’ve got to go this way or that way.” Your circumstances are unpredictable, too. It could be, “We’ve got this goal.” But then business collapses, and we change the goal. Or even if you’ve got the same goal, we have to adjust the target. There’s all kinds of stuff that’s in play, so it’s not really a contract. It’s kind of a relationship.

One of the questions that we wanted to look at was to what extent is a performance appraisal a contract, and to what extent is it a relationship where it is used to encourage you? We also wanted to see some basic things. There are some people who claim that it really doesn’t drive very much about your outcomes. Merit pay is based on something else. It’s about bias or how the company is doing, and that if you get cozy with your supervisor, you get good appraisals. If you don’t, then you get bad ones. But here’s maybe the biggest thing, which we weren’t so interested in academically when we started this. But practically, it’s really important. Do people who perform well always perform well? And people who perform poorly, do they always perform poorly? The reason this matters is because there is a very prominent theory in the practice of management — something that Jack Welch made famous — about the A-player, B-player, C-player model. The folks at McKinsey & Co. were making a similar case that there are really good executives and there are kind of lousy ones. The big thing you want to do if you believe that is to hire the good ones and get rid of the bad ones. If that’s the story, then management’s kind of simple, right? You just hire the good people, screen them and see how they do. If they do bad, out they go.

As far as we can tell, no one has ever looked at this before or at least published it. Are the people who do well always doing well, or not? If we know your scores this year for everybody in the company, how much of next year’s score could we predict or explain? If the good people are always good and the bad people are always bad, we can explain 100% of your scores because next year’s score will be identical to this year’s score. If it’s random, which would be kind of astonishing, then it would be zero. There’d be no relationship between how people on average perform this year and how they perform next year. The good people could be good, the bad people could be good or bad.

Knowledge@Wharton: But you would think that they would follow a pattern. If you’re good in 2014, unless something has drastically changed, you’re going to be pretty good in 2015 as well.

Cappelli: Right. It’s between zero and 100%. If you think this A-player, B-player, C-player model is right, it’s going to be closer to 100. If you think it’s all just random or it’s kind of noise or people vary a lot, you’re closer to zero. So, that’s the question.

Knowledge@Wharton: I’m going to say that it would probably be closer to 70 or 75.

Cappelli: That’s a very common answer. People in human resources guess 80%. The correct answer is 27%, so it’s way closer to zero than it is to 100%.

Knowledge@Wharton: Why so much lower? I would think that it would be almost an automatic that it would be on the higher end.

Cappelli: Many people seem to believe that, especially people in human resources. But when I ask them if they have ever actually looked at it, the answer is no. They just assume it’s that way. Maybe they assume it’s that way because that’s what you hear from the A-player, B-player, C-player kind of story, and you could see some of this is a cognitive bias. There’s something in psychology known as the fundamental attribution error. It means that when you see somebody behave in a particular way, we are inclined to assume it is because of who they are rather than the circumstances. The classic example is somebody racing by you on the expressway going home. They’re driving on the shoulder and whipping past. Your inclination is to say, “That guy’s a jerk,” rather than to even entertain the idea that maybe it’s an emergency. We seem to be wired to think everything is due to the person. If you believe that, then you would be inclined to think the A-player, B-player, C-player model is right and good players this year are going to be good players next year, etc.

The other thing we looked at was to see whether it actually changed your appraisal scores when you got a new supervisor, because the other view is that you get comfy with a particular supervisor, then your scores are always sort of the same. You get a new supervisor, and they can really sort out whether you’re good or bad. Well, we didn’t see that either.

Knowledge@Wharton: I can see that happening on both sides. Either you have a supervisor that you just don’t get along with right from the start and your performance appraisal would be lower, or it could be the same if you get along with somebody.

Cappelli: We didn’t go in with a prior expecatation, saying, “Gosh, this is silly to think that it’s all disposition.” Or, “Boy, it’s almost all random.” We had no idea what we were going to find. In fact, we were looking at that as a way to test the real things we thought were going to be interesting, which was is it more like a contract or not? It turns out there’s a lot of variation in how people perform. One of the things that calls into question are these forced ranking systems, or they call them “rank and yank” or “rack and whack.” General Electric used to force out the bottom 10% because they believed it was the A-player, B-player, C-player model. If your company’s doing that, you might want to actually look to see whether it’s true that your bottom 10% this year are the same as your bottom 10% next year. The problem is, if you keep firing your bottom 10%, you’re never going to know because you’ll never know what those guys would have done. But you could at least look at the appraisal scores for everybody else and see whether they remain constant over time. If they’re bouncing around a lot, it is insane to fire the bottom 10% because there’s no reason to think those guys are going to be bad next year.

Knowledge@Wharton: Is that 50^th percentile kind of the perfect area?

Cappelli: I don’t know what is good or bad out of this. I think if you believe management matters, you would like to think that it’s not a perfect correlation. You’d like to think that the numbers are a little lower because you could shape it. You could take the same person with a different manager, a different context, and they could perform differently — better or worse. I think it’s encouraging to management as a field that the relationship is lower. But it makes it harder for people running businesses and employers because now it’s not just picking the good people and then getting out of their way.

Knowledge@Wharton: But performance appraisals have seemingly taken on more importance in the last 20, 30 years because the elements of psychology now are factored into business so much. Companies want to know what their employees are thinking, more so than ever before.

Cappelli: Honestly, there was a high watermark of that stuff about 40 years ago or so. For example, AT&T had a team of about 15 psychologists, through 1980, that just tinkered with the performance appraisal form every year. And in the 1960s, performance appraisals were so thorough that you were assessed on the appraisals you gave your subordinates. They would read your appraisals, and if you didn’t do a good job, it affected your appraisal. They’d also see how your subordinates did years later. If they did better in their careers, that affected your own appraisal. They used to take this stuff way more seriously, and we don’t anymore.

Knowledge@Wharton: It has been pared down because of what factors?

Cappelli: There are a couple. The first one is that we’ve given supervisors a ton of other stuff to do. It used to be that your job was to supervise people and that was it. Now, you’re an individual contributor, and you’re supervising these folks. The second thing that’s happened is the span of control has increased. That means a number of people reporting to you. There used to be a rule that six or seven were the most people you ought to supervise. Now, it’s up in the 20s in lots of places. If you’re trying to follow 20 employees and pay attention to what they’re doing and be an individual contributor, it’s almost impossible to pay much attention to them.

Knowledge@Wharton: Did you look at employees involved in sales whose performance could be measured objectively?

Cappelli: We looked at a retail organization, and we could see the store managers. They had 10 attributes that they were assessed on, and I think six of them were actually hard numbers — financial performance, store sales, things like that. So, they were objective numbers, and the performance bounced around a lot. We couldn’t tell who the manager was and go in and interview that manager. As you’d imagine, there were hundreds of managers, so it would have been pretty hard to do. As we were saying before, the best predictors seem to be things which have more to do with bias than with good management practice. Although you would think that good management practice ought to matter. It’s just a little hard to measure in a study.

Knowledge@Wharton: You mentioned that in certain situations that the boss is asking the employee questions such as, “What kind of job did I do over the course of the year?”

Cappelli: You know, 360-degree feedback is the formal way in which that gets done, where you ask people all the way around you, “How do you think I did as a boss?” That has not had a terrific track record, partly because there’s a lot of venting going on. If you’re a subordinate, it’s hard to be objective about your boss. It’s hard, always, to like your boss.

Let me tell you the punch line of what we found on the academic side: Things don’t look very much like a contract, and supervisors tend to reward people for improvements as well as the level of performance. So, if you’re doing better this year than last year, it’s not like, “You did well. You get this much.” Also, counter to the prevailing view, they over-reward high performers. It’s not a linear relationship. If you’re a poor performer, they really do whack your merit pay increases. And if you’re a better performer, they really do load them up.

Knowledge@Wharton: And that increases that separation between the upper end and the lower end, in the course of the job.

Cappelli: Within the job, that’s right. And it is true that as you move up the organization, the average scores increase. Why is that? There’s two explanations. One is that you’re selecting better people if it’s promotion from within. So, it’s not surprising that the scores would go up. And is it bias once you get near the top? They say that CEOs always give their personal assistants the top score.

http://knowledge.wharton.upenn.edu/article/the-problem-with-job-performance-appraisals/?utm_source=kw_newsletter&utm_medium=email&utm_campaign=2016-05-25