In their new book Innovation Accounting: A Practical Guide for Measuring Your Innovation Ecosystem’s Performance, authors Dan Toma and Esther Gons lay out nine of the biggest myths related to trying to measure innovation – from tallying R&D expenditures to waiting until a new concept has been launched to try to gauge its impact.
“Making decisions rooted in myths can be bad for business,” they write. Read the excerpt below, or get the full book for more insights on putting a solid innovation accounting regime in place.
The Myths of Measuring Innovation
A number of common myths tend to inform whether or how a company attempts to measure innovation. Unfortunately, these myths are only loosely tied to the real world; they don’t reflect reality. Making decisions rooted in myths can be bad for business. We’ll now do our best to debunk some of the more common myths around measuring innovation.
Myth #1: R&D expenditure is a good indicator of innovation
It seems logical for companies to equate R&D spending with innovation, or to believe that the amount of money a company puts into R&D correlates with how innovative that company is. The thing is, spending, even spending on research, and innovation are not the same things. And while a kind of common sense equates R&D with innovation, the data tells a different story.
Strategy & Business, a unit within PricewaterhouseCoopers, has published an annual report of the 1000 most innovative companies in the world for more than twelve years running. In that time, they have found no statistically significant relationship between R&D spending and sustained financial performance.
This finding applies to total R&D spend, as well as R&D spending as a percentage of revenues. Spending on R&D is not related to growth in sales or profits, increases in market capitalization, or shareholder returns. In every annual report that Strategy & Business has published, the 10 most innovative companies are often not the top 10 spenders on R&D.
What R&D spending seems to generate is an increase in the number of patents held by a company. And while having more patents can mean more possibilities, sitting on a library of unused patents is nowhere near what it means to be innovative. If every patent represents new applications and opportunities, then simply amassing them without bringing them to market, or failing to bring them to market, in some ways represents a strange kind of complacency with the status quo.
If every patent represents new applications and opportunities, then simply amassing them without bringing them to market, or failing to bring them to market, in some ways represents a strange kind of complacency with the status quo.
Consider this: In the late 1970s, Xerox invented the computer mouse in their Palo Alto Research Center (PARC). And more than that, they invented a machine that could use it. Up to then, interacting with a computer meant typing in key commands. With this new invention though, users could click and drag, open and close windows, and interface with computers in a whole new way.
So why isn’t Xerox widely known for being the company behind this technology? Partially because they couldn’t recognize their own brilliance, and partially because someone else, a young entrepreneur by the name of Steve Jobs, could. At least that’s how the legend goes. One day, Jobs visited PARC. A Xerox engineer demoed the mouse for Jobs, and he was blown away. So much so, that he ran back to Apple and demanded a mouse, and windows, and a new way of interacting with a computer too. After some time, Apple developed the Macintosh, and the rest is history.
While a number of people have disputed the particulars of this story; whether Jobs visited PARC once or twice, whether Apple was researching similar technology at the same time or not, the legend still teaches an important lesson: Innovation doesn’t just mean securing patents or coming up with new ideas, it also means doing something with them. Had Xerox been able to turn its developmental genius into commercial success, Jobs said, “it could have been as big as I.B.M. plus Microsoft plus Xerox combined—and the largest high-technology company in the world.”
Myth #2: Innovation can’t be measured because innovation is creativity and creativity can’t be measured
There is a common tendency to conflate creativity with innovation. Management often sees successful startups coming up with great new products, which motivates managers to pursue the development of similarly cool, new, shiny products.
Companies fall into this trap every time. They think that the best way to become better at innovation is to put a bunch of “creatives” under one roof and ask them to think of the next big thing. To be clear, awesome ideas are an important input to innovation. But just putting a bunch of subjectively creative individuals together isn’t a foolproof recipe for generating those ideas. What’s more, if companies stop at idea-generation, they are in for a huge disappointment. As Scott Anthony writes in The Little Black Book of Innovation, “Innovation is a process that combines discovering an opportunity, blueprinting an idea to seize that opportunity, and implementing that idea to achieve results. Remember: no impact, no innovation.”
Innovation is a discipline to be mastered and managed. And it is hard work! But if innovation is more about discipline and routine than about creativity, it can be measured, just as any other company process can.
Innovation is a discipline to be mastered and managed… if innovation is more about discipline and routine than about creativity, it can be measured, just as any other company process can.
Myth #3: The success of a new innovative venture can only be measured once it’s in the market
Nothing could be further from the truth. There are many different signals along the life of an idea; signals that can indicate if an idea is heading in the right direction or not.
Say, for example, a team is working on a new product to solve a very specific problem. The team interviews potential customers about their experience with this problem, but no one they speak to seems to struggle with this issue. It seems, at least based on these interviews, that the team is solving a non-problem. If this signal isn’t noticed and acted upon, the team might end up building a “solution” for a market that doesn’t exist.
Understanding these signals and responding to them appropriately is how venture capital investors make their living. Corporations that seek to grow through innovation need to start training their managers to recognize and respond to these kinds of signals.
Myth #4: Everything is a KPI
No matter how relevant or insignificant an indicator is, they all tend to be referred to as Key Performance Indicators (KPIs). However, there is a difference between Key Performance Indicators (KPIs) and Key Result Indicators (KRIs). Performance indicators are non-financial in nature and they can be traced back to the activity the innovation team has performed day-in and day-out (e.g. churn rates, number of coaching hours booked for next month, number of customer interviews booked for next month, etc.). Result indicators give a broad summary of what happened as a result of the process (e.g. net profit, return on capital employed, cumulated customer satisfaction, employee satisfaction, etc.).
Myth #5: Measuring innovation means measuring the number of ideas in the company
Measuring ideas is an essential part of an innovation accounting system. Without ideas upstream, a company can hardly expect any returns downstream. But just measuring the number of ideas the company has in a backlog or the number of projects currently under development is not nearly enough.
Some companies measure product teams, but don’t measure their entire portfolio. Others measure the number of ideas that their company initiates, but not how many of those ideas are still alive after two years, or what their adoption rate is in existing business units. That’s why so many companies get a partial view. They focus intensely on specific business units or certain metrics, and they get a picture of what’s going on; just not the whole picture.
Measuring a company’s innovation ecosystem requires a wider view. In other words, it means measuring the whole, not just a few chosen parts.
Measuring a company’s innovation ecosystem requires a wider view. In other words, it means measuring the whole, not just a few chosen parts.
Myth #6: All innovation measurements work successfully in any organization
Contrary to popular belief, not all innovation measurements can work successfully in any organization. Why? First, companies vary a lot (a B2B company is pretty different from a B2B2C one). And second, for each company, innovation has a different meaning and importance. What it means to innovate in a B2B software company is not the same as what it means to innovate in an online, direct to consumer, business. Innovating in hospitality is something else entirely.
While the principles of Innovation Accounting are universal, the reality is that innovation measurements themselves need to be contextualized; they must reflect the peculiarities and particularities of the company using them. They can’t just be copy-pasted from one company to another.
Myth #7: All innovation measurements work successfully for any type of innovation
People tend to think of “innovation” as one thing, when, in reality, innovation takes more than one form. Incremental innovation, the little-by-little changes that improve existing products and processes, is different from breakthrough innovation, which can upend entire industries seemingly overnight. Different kinds of innovation need to be measured differently. For incremental innovation, metrics need to focus on questions of efficiency: how efficient are projects going through the pipeline? And how are they stacking up against forecasted and actual economic return? For breakthrough innovation, given the inherent uncertainty of something that’s new, the measurements must be different.
Myth #8: Innovation measurements are only needed to relieve executive and stakeholder anxiety around investing in innovation
We speak on the topic of Innovation Accounting at various events and conferences worldwide. And after every talk we give, almost like clockwork, someone from the audience approaches us, asking for help with some innovation measurements that would help them justify the existence of their team or lab to a superior.
The idea that measurements only exist to convince management is probably one of the biggest myths. At the same time, it’s also one of the biggest pitfalls of Innovation Accounting.
While innovation measurements are important for executives and stakeholders, they should not be designed exclusively for their needs. When they are, they tend to focus narrowly on the financial aspect of innovation. But innovation measurements should reflect the whole innovation ecosystem and all of its processes, not just a single part (here, financial metrics). At the end of the day, innovation measurements should lead to changes in behavior; they should serve as the basis for continuous improvements.
Myth #9: Tying innovation measurements to incentives means more and/or better outcomes
It is a myth that money is the primary driver for staff, and that, in order to achieve better performance, organizations must design financial incentives. Recognition, respect, and self-actualization are more important drivers. In all types of organization, there is a tendency to believe that the way to make KPIs work is to tie those KPIs to an individual’s pay. But when KPIs are tied to bonuses, they are more likely to be gamed for personal benefit.
Every KPI can be gamed, but the ones tied to financial gains are more susceptible than others. Every measurement has a dark side, a negative consequence, or an unintended action that can lead to inferior performance.
For example, one time we were coaching innovation teams in an engineering company’s accelerator. Knowing the importance of lean experimenting, we said we wanted to incentivize teams to perform more experiments. As soon as we told the teams we would be measuring the number of experiments they performed each week, they began to claim that everything they did was an experiment. At that point, we had a whole other issue on our hands. We had to sit down with every team to see which of the things they’ve performed were really experiments and which ones weren’t. This ended up being a huge time suck for us.
And this doesn’t just happen in engineering. Another time, case workers in a government agency were told they’d be measured on cases closed. The result? Experienced workers flew through large numbers of easy cases, leaving the difficult ones to the inexperienced staff. Not exactly the desired outcome.
Every measurement has a dark side, a negative consequence, or an unintended action that can lead to inferior performance.
Of course, the end of our list doesn’t mean the end of all myths around innovation measurements. There are certainly many others. This list was based on our experience interfacing with corporate managers trying to measure innovation in their own companies. In other cases, we simply saw managers using these myths to shy away from trying to measure innovation at all.
Regardless of the situation, failing to measure innovation means having no data or evidence to support innovation efforts. And having no data or evidence to support innovation efforts often means those efforts get shelved, regardless of the promise they might have held.
