Ten red flags signaling your analytics program will fail
Struggling
to become analytics-driven? One or more of these issues is likely what’s
holding your organization back.
These days, it’s the
rare CEO who doesn’t know that businesses must become analytics-driven. Many
business leaders have, to their credit, been charging ahead with bold
investments in analytics resources and artificial intelligence (AI). Many CEOs have dedicated a lot of their own time to
implementing analytics programs, appointed chief analytics officers (CAOs) or
chief data officers (CDOs), and hired all sorts of data specialists.
However, too many executives
have assumed that because they’ve made such big moves, the main challenges to
becoming analytics-driven are behind them. But frustrations are beginning to
surface; it’s starting to dawn on company executives that they’ve failed to
convert their analytics pilots into scalable solutions. (A recent McKinsey
survey found that only 8 percent of 1,000 respondents with analytics
initiatives engaged in effective scaling practices.) More boards and
shareholders are pressing for answers about the scant returns on many early and
expensive analytics programs. Overall, McKinsey has observed that only a small
fraction of the value that could be unlocked by advanced-analytics approaches
has been unlocked—as little as 10 percent in some sectors. And McKinsey’s AI Index reveals that the gapbetween leaders and laggards in
successful AI and analytics adoption, within as well as among industry sectors,
is growing.
That said, there’s one
upside to the growing list of misfires and shortfalls in companies’ big bets on
analytics and AI. Collectively, they begin to reveal the failure patterns
across organizations of all types, industries, and sizes. We’ve detected what
we consider to be the ten red flags that signal an analytics program is in
danger of failure. In our experience, business leaders who act on these alerts
will dramatically improve their companies’ chances of success in as little as
two or three years.
1. The executive team doesn’t have a clear vision for its
advanced-analytics programs
In our experience, this
often stems from executives lacking a solid understanding of the difference
between traditional analytics (that is, business intelligence and reporting)
and advanced analytics (powerful predictive and prescriptive tools such
as machine learning).
To illustrate, one
organization had built a centralized capability in advanced analytics, with heavy
investment in data scientists, data engineers, and other key digital roles. The
CEO regularly mentioned that the company was using AI techniques, but never
with any specificity.
In practice, the
company ran a lot of pilot AI programs, but not a single one was adopted by the
business at scale. The fundamental reason? Top management didn’t really grasp
the concept of advanced analytics. They struggled to define valuable problems
for the analytics team to solve, and they failed to invest in building the
right skills. As a result, they failed to get traction with their AI pilots.
The analytics team they had assembled wasn’t working on the right problems and
wasn’t able to use the latest tools and techniques. The company halted the
initiative after a year as skepticism grew.
First response: The CEO, CAO, or CDO—or
whoever is tasked with leading the company’s analytics initiatives—should set
up a series of workshops for the executive team to coach its members in the key
tenets of advanced analytics and to undo any lingering misconceptions. These
workshops can form the foundation of in-house “academies” that can continually
teach key analytics concepts to a broader management audience.
2. No one has determined the value that the initial use
cases can deliver in the first year
Too often, the
enthusiastic inclination is to apply analytics tools and methods like
wallpaper—as something that hopefully will benefit every corner of the
organization to which it is applied. But such imprecision leads only to
large-scale waste, slower results (if any), and less confidence, from
shareholders and employees alike, that analytics initiatives can add value.
That was the story at a
large conglomerate. The company identified a handful of use cases and began to
put analytics resources against them. But the company did not precisely assess
the feasibility or calculate the business value that these use cases could
generate, and, lo and behold, the ones it chose produced little value.
First response: Companies in the early stages
of scaling analytics use cases must think through, in detail, the top three to
five feasible use cases that can create the greatest value quickly—ideally
within the first year. This will generate momentum and encourage buy-in for
future analytics investments. These decisions should take into account impact,
first and foremost. A helpful way to do this is to analyze the entire value
chain of the business, from supplier to purchase to after-sales service, to
pinpoint the highest-value use cases.
To consider
feasibility, think through the following:
·
Is the data needed for
the use case accessible and of sufficient quality and time horizon?
·
What specific process
steps would need to change for a particular use case?
·
Would the team involved
in that process have to change?
·
What could be changed
with minimal disruption, and what would require parallel processes until the
new analytics approach was proven?
3. There’s no analytics strategy beyond a few use cases
In one example, the
senior executives of a large manufacturer were excited about advanced
analytics; they had identified several potential cases where they were sure the
technology could add value. However, there was no strategy for how to generate
value with analytics beyond those specific situations.
Meanwhile, a competitor
began using advanced analytics to build a digital platform, partnering with
other manufacturers in a broad ecosystem that enabled entirely new product and
service categories. By tackling the company’s analytics opportunities in an
unstructured way, the CEO achieved some returns but missed a chance to
capitalize on this much bigger opportunity. Worse yet, the missed opportunity
will now make it much more difficult to energize the company’s workforce to
imagine what transformational opportunities lie ahead.
As with any major
business initiative, analytics should have its own strategic direction.
First response: There are three crucial
questions the CDO or CAO must ask the company’s business leaders:
·
What threats do
technologies such as AI and advanced analytics pose for the company?
·
What are the
opportunities to use such technologies to improve existing businesses?
·
How can we use data and
analytics to create new opportunities?
4. Analytics roles—present and future—are poorly defined
Few executives can
describe in detail what analytics talent their organizations have, let alone
where that talent is located, how it’s organized, and whether they have the
right skills and titles.
In one large
financial-services firm, the CEO was an enthusiastic supporter of advanced
analytics. He was especially proud that his firm had hired 1,000 data
scientists, each at an average loaded cost of $250,000 a year. Later, after it
became apparent that the new hires were not delivering what was expected, it
was discovered that they were not, by strict definition, data scientists at
all. In practice, 100 true data scientists, properly assigned in the right
roles in the appropriate organization, would have sufficed. Neither the CEO nor
the firm’s human-resources group had a clear understanding of the
data-scientist role—nor of other data-centric roles, for that matter.
First response: The right way to approach the
talent issue is to think about analytics talent as a tapestry of skill sets and
roles (interactive). Naturally, many of these capabilities and roles
overlap—some regularly, others depending on the project. Each thread of that
tapestry must have its own carefully crafted definition, from detailed job
descriptions to organizational interactions. The CDO and chief human resources
officer (CHRO) should lead an effort to detail job descriptions for all the
analytics roles needed in the years ahead. An immediate next step is to
inventory all of those currently with the organization who could meet those job
specifications. And then the next step is to fill the remaining roles by hiring
externally.
5. The organization lacks analytics translators
If there’s one
analytics role that can do the most to start unlocking value, it is the analytics translator.
This sometimes overlooked but critical role is best filled by someone on the
business side who can help leaders identify high-impact analytics use cases and
then “translate” the business needs to data scientists, data engineers, and
other tech experts so they can build an actionable analytics solution.
Translators are also expected to be actively involved in scaling the solution
across the organization and generating buy-in with business users. They possess
a unique skill set to help them succeed in their role—a mix of business knowledge,
general technical fluency, and project-management excellence.
First response: Hire or train translators
right away. Hiring externally might seem like the quickest fix. However, new
hires lack the most important quality of a successful translator: deep company
knowledge. The right internal candidates have extensive company knowledge and
business acumen and also the education to understand mathematical models and to
work with data scientists to bring out valuable insights. As this unique
combination of skills is hard to find, many companies have created their own
translator academies to train these candidates. One global steel company, for
example, is training 300 translators in a one-year learning program. At
McKinsey, we’ve created our own academy, training 1,000 translators in the past
few years.
6. Analytics capabilities are isolated from the business,
resulting in an ineffective analytics organization structure
We have observed that
organizations with successful analytics initiatives embed analytics
capabilities into their core businesses. Those organizations struggling to
create value through analytics tend to develop analytics capabilities in
isolation, either centralized and far removed from the business or in sporadic
pockets of poorly coordinated silos. Neither organizational model is effective.
Overcentralization creates bottlenecks and leads to a lack of business buy-in.
And decentralization brings with it the risk of different data models that
don’t connect.
A definite red flag
that the current organizational model is not working is the complaint from a
data scientist that his or her work has little or no impact and that the
business keeps doing what it has been doing. Executives must keep an ear to the
ground for those kinds of complaints.
First response: The C-suite should consider a
hybrid organizational model in which agile teams combine talented professionals from both the business side
and the analytics side. A
hybrid model will retain some centralized capability and decision rights
(particularly around data governance and other standards), but the analytics
teams are still embedded in the business and accountable for delivering impact.
For many companies, the
degree of centralization may change over time. Early in a company’s analytics
journey, it might make sense to work more centrally, since it’s easier to build
and run a central team and ensure the quality of the team’s outputs. But over
time, as the business becomes more proficient, it may be possible for the
center to step back to more of a facilitation role, allowing the businesses
more autonomy.
7. Costly data-cleansing efforts are started en masse
There’s a tendency for
business leaders to think that all available data should be scrubbed clean
before analytics initiatives can begin in earnest. Not so.
McKinsey estimates that
companies may be squandering as much as 70 percent of their data-cleansing
efforts. Not long ago, a large organization spent hundreds of millions of
dollars and more than two years on a company-wide data-cleansing and
data-lake-development initiative. The objective was to have one data
meta-model—essentially one source of truth and a common place for data
management. The effort was a waste. The firm did not track the data properly
and had little sense of which data might work best for which use cases. And
even when it had cleansed the data, there were myriad other issues, such as the
inability to fully track the data or understand their context.
First response: Contrary to what might be
seen as the CDO’s core remit, he or she must not think or act “data first” when
evaluating data-cleansing initiatives. In conjunction with the company’s
line-of-business leads and its IT executives, the CDO should orchestrate data
cleansing on the data that fuel the most valuable use cases. In parallel, he or
she should work to create an enterprise data ontology and master data model as
use cases become fully operational.
8. Analytics platforms aren’t built to purpose
Some companies know
they need a modern architecture as a foundation for their digital
transformations. A common mistake is thinking that legacy IT systems have to be
integrated first. Another mistake is building a data lake before figuring out
the best ways to fill it and structure it; often, companies design the data
lake as one entity, not understanding that it should be partitioned to address
different types of use cases.
In many instances, the
costs for such investments can be enormous, often millions of dollars, and they
may produce meager benefits, in the single-digit millions. We have found that
more than half of all data lakes are not fit for purpose. Significant design
changes are often needed. In the worst cases, the data-lake initiatives must be
abandoned.
That was the case with
one large financial-services firm. The company tried to integrate its legacy
data warehouses and simplify its legacy IT landscape without a clear business
case for the analytics the data would fuel. After two years, the business began
to push back as costs escalated, with no signs of value being delivered. After
much debate, and after about 80 percent of the investment budget had been spent,
the program screeched to a halt.
First response: In practice, a new data platform can exist in parallel with legacy systems. With appropriate input from the
chief information officer (CIO), the CDO must ensure that, use case by use
case, data ingestion can happen from multiple sources and that data cleansing
can be performed and analytics conducted on the platform—all while the legacy
IT systems continue to service the organization’s transactional data needs.
9. Nobody knows the quantitative impact that analytics is
providing
It is surprising how
many companies are spending millions of dollars on advanced analytics and other
digital investments but are unable to attribute any bottom-line impact to these
investments.
The companies that have
learned how to do this typically create a performance-management framework for
their analytics initiatives. At a minimum, this calls for carefully developed
metrics that track most directly to the initiatives. These might be
second-order metrics instead of high-level profitability metrics. For example,
analytics applied to an inventory-management system could uncover the drivers
of overstock for a quarter. To determine the impact of analytics in this
instance, the metric to apply would be the percentage by which overstock was
reduced once the problem with the identified driver was corrected.
Precisely aligning
metrics in this manner gives companies the ability to alter course if required,
moving resources from unsuccessful use cases to others that are delivering
value.
First response: The business leads, in
conjunction with translators, must be the first responders; it’s their job to
identify specific use cases that can deliver value. Then they should commit to
measuring the financial impact of those use cases, perhaps every fiscal
quarter. Finance may help develop appropriate metrics; the function also acts
as the independent arbiter of the performance of the use cases. Beyond that,
some leading companies are moving toward automated systems for monitoring
use-case performance, including ongoing model validation and upgrades.
10. No one is hyperfocused on identifying potential
ethical, social, and regulatory implications of analytics initiatives
It is important to be
able to anticipate how digital use cases will acquire and consume data and to
understand whether there are any compromises to the regulatory requirements or
any ethical issues.
One large industrial
manufacturer ran afoul of regulators when it developed an algorithm to predict
absenteeism. The company meant well; it sought to understand the correlation
between job conditions and absenteeism so it could rethink the work processes
that were apt to lead to injuries or illnesses. Unfortunately, the algorithms
were able to cluster employees based on their ethnicity, region, and gender,
even though such data fields were switched off, and it flagged correlations
between race and absenteeism.
Luckily, the company
was able to pinpoint and preempt the problem before it affected employee
relations and led to a significant regulatory fine. The takeaway: working with
data, particularly personnel data, introduces a host of risks from algorithmic bias.
Significant supervision, risk management, and mitigation efforts are required
to apply the appropriate human judgment to the analytics realm.
First response: As part of a well-run broader
risk-management program, the CDO should take the lead, working with the CHRO
and the company’s business-ethics experts and legal counsel to set up
resiliency testing services that can quickly expose and interpret the secondary
effects of the company’s analytics programs. Translators will also be crucial
to this effort.
There is no time to
waste. It is imperative that businesses get analytics right. The upside is too
significant for it to be discretionary. Many companies, caught up in the hype,
have rushed headlong into initiatives that have cost vast amounts of money and
time and returned very little.
By identifying and
addressing the ten red flags presented here, these companies have a second
chance to get on track.
By Oliver Fleming, Tim Fountaine, Nicolaus Henke, and Tamim Saleh
https://www.mckinsey.com/business-functions/mckinsey-analytics/our-insights/ten-red-flags-signaling-your-analytics-program-will-fail?cid=other-eml-alt-mip-mck-oth-1805&hlkid=87928bc1769a40c5b59cdf78b451bda5&hctky=1627601&hdpid=ec2a5899-4158-4206-a1a4-8e4a5ab1d49e