Can Economists ruin Innovation with an Index? Thoughts on measuring innovation, growth, culture, and creativity.
The 2022 Global Innovation Index report released by WIPO has been published for 2022. Here are some of thoughts on the report, including a few comments and reflections on innovation, economic growth, measurement, and cultural resources.
As I’ve written about the global innovation index when it was released last year (Here’s a link to last year’s comments & report), I thought that it would be relevant to update this with the new report and comments on it.
The Global Innovation Index – Background
The report is meant as an overview of global innovation. This was described in detail before (Here and Here), so will not repeat the definitions, but will just highlight a couple of starting points:
It reveals the most innovative economies in the world, ranking the innovation performance of around 132 economies while highlighting innovation strengths and weaknesses.
As defined on the WIPO website
The Global Innovation Index (or GII), “Captures the Innovation Ecosystem Performance of 132 countries”.
The previous post referenced (link) contains some basic information on the rationale, areas, and the composition of the measure.
Measuring Innovation: The Spookiness of Composite Measures & Abstract Art
This is a screenshot from the report, showing the components of the composite index. (What is a composite measure?).
The Blue are innovation inputs, and the purple are outputs. Each measure itself is made up of further sub-measures :
There have been some cosmetic changes, but the Innovation Index is understood by measuring 8 (in turn composite) variables:
The five input measures:
- Institutions,
- Human Capital and Research,
- Infrastructure,
- Market Sophistication,
- Business Sophistication
and two output measures:
- Knowledge and Technology outputs, and
- Creative outputs
My intuition is, at this scale, this is very problematic. Look at all these different measures, counts, and surveys that constitute the sub-measures. How can it not contain so much noise! Considering all these variables, it would be quite hard to argue that the innovation index (at best) isn’t transformed into some non-random “Noise” reflecting an arbitrary ratio of GDP and Economy variables, distorted by equally arbitrary peculiarities of the countries and their social conditions?
How many of these are simply the result of GDP, making this correlation a dominant effect…?
The problem is that there is no way of really telling.
Sample problematic measures
Look at this variable (part of the business environment / institutions sub-measure), which is the substance of nightmares (not the actual measure, but its use here!) :
1.3.2. Entrepreneurship policies and culture*
Entrepreneurship policies and culture index* | 2021
Average perception scores (five-year average) of experts on entrepreneurial policies and entrepreneurial culture (Items B, C and I3 and I4 of the GEM National Expert Survey).
Experts in different fields (purposive sampling, minimum 36 experts per year) assess conditions for entrepreneurship in their country via statements (1= completely false; 10 = completely true). Country participation in GEM varies and therefore the number of experts and years on which this item is based differs according to country.
This is somehow as scary:
5.2.1. University–industry R&D collaboration†
The extent to which businesses and universities collaborate on R&D† | 2021 Average answer to the survey question: In your country, to what extent do businesses and universities collaborate on research and development (R&D)? [1 = not at all; 7 = to a great extent]
and consider this:
6.2.1. Labor productivity growth, %
Growth rate of GDP per person employed (%, five-year average) | 2021
Growth rate of real GDP per person employed, average of five most recent available years (2017–2021). Growth of GDP per person engaged provides a measure of labor productivity (defined as output per unit of labor input). GDP per person employed is GDP divided by total employment in the economy
–> This variable is less nightmarish, but I included it here as a sample on noise: so if a country was rich in Durbanium (an arbitrary non-existent metal), and Durbanium prices spiked one year, this variable will shoot up, and so will that country’s Index rank… Durbanium shortages made this country more innovative!
Note – some other measures are just as – if not more – horrific. Go through the doc below, and imagine the possible origins of some apparent increases in the Innovation Index… (this is a recreational activity).
Some Timely Highlights: Investing in Innovation and Economic Growth
The report mentions that 2021 witnessed a huge growth in VC activity. This would be interesting to look at as steps to curb inflation by different central banks, led by the US FED will lead to reduction of ‘available money’ by increasing borrowing costs.
This year’s GII finds the innovative sectors of the world
From the Report
economy at a crossroads. On the one hand, science
and innovation investments continued to surge in
2021, performing strongly even at the height of a once
in a century pandemic. International patent filings,
R&D expenditure, scientific publications and other key
innovation metrics also all showed continued growth.
Take the trend in venture capital (VC) deals. Typically,
the pool of capital available for financing innovation
shrinks during periods of economic turbulence, with
VC investment declining in line with the overall business
cycle. However, the current crisis has instead seen a
historic boom in VC activity, with the number of deals
increasing by almost 50 per cent last year.
This concept of free money and rampant inflation leading to increases in levels of VC activity (Even Neumannn seems to have received a reward for his WeWork genius), and thus apparent improvement in innovation readiness, takes us to the eternal problem of quantifying innovation-driven growth. Which innovation is causing/leading to economic growth, and how much..?
Innovation investments ‘thrived’ because of all the free money (apparently), and this extra money in the systems will likely lead to higher inflation and a number of central banks measure, which will eventually lead to a reduction in economic growth (obviously). So in effect, this increase in funds is just a signal that soon we’ll have lower economic growth, making the attribution problem even worse!
This somehow shows the failure of compartmentalized and reductionist theorizing from a limited perspective, especially in dealing with complex and interconnected phenomena like an innovation ecosystem. Interestingly, using the word ecosystem might create a (false) sense of satisfaction and security about all the terrifying reductionism a certain model might be committing.
A key difficulty here is the lag between any investment (or active effort) and results, as well as the problem of attribution (what did to what). This is one of the greatest challenges for managers, but more so for policy makers, especially when trying to reconcile long-term results with current actions. How to incorporate the lag into the measures, and how to to include more grounded and action-focused themes.
A better Innovation Index : A case for prioritizing Specifics & “Innovation in identifying Innovation” VS “Country Rankings”
I understand that one of the motivations for the team was to create a global measure, but I just can’t get past my problem with the focus on “ranking”.
“Ranking” is the star of the index. But let’s give it some thought: Country X is number 32, and china climbed three places, and france dropped two and someone consolidated their position…
What does all this really mean? Does the “Race” model really fit (where convenient)?
What are the implications and motivations for that?
“Ranking” goes with the composite measure well… but that just makes some aspects of the work more questionable.
I imagine it would have been really beneficial to replace the section on rankings, and the reduction of efforts to blind numbers with something (much) more useful. What about some stories and qualitative discussions? Success stories from different countries related to specific areas of problems? Ways of dealing with specific innovation-crippling problems (what are those?)? What about longer term variable associations? Are there specific cases where institutional changes have ushered the creation of new industries or segments?
What is really annoying that the talk of ranking countries precedes and eclipses the discussion of the validity and reliability of the indicator, and what it really means to be an innovator… An index on innovation would gain a lot from a more human approach that really and deeply considers all the dimensions of innovation… but… Economists!
Cultural Resources, Cultural Industries, and an Innovation Index
As I’ve written before, for many countries, relying on their cultural and creative industries, and utilizing their different cultural resources could be an excellent way to advance innovation and promote innovation-driven economic growth (See this post and article on the cultural innovation subsystem: link).
This can happen through many pathways, and can lead to the development of a few central industries… Importantly it might be more feasible and less risky than aiming for R&D / S&T based innovation without having proper infrastructures.
I feel that there is a growing awareness of this, and this is (to a small degree) reflected in the report, particularly the section on creative outputs… Much more work can (and needs to) be done here.
Documents : The GII 2022 Report + The definitions Appendix
The full report can be downloaded here :
The full list of definitions of items in the Index can be found here :
Funny bit : When does it become “not so interesting” to classify 🙂 🙂 … They are not “below average” they are “all the others” .
Concluding thoughts and Recommendations: Fuzziness, Approximation, Measurement, and Stories
The approach to try and quantify and measure innovation here is very educational. Different layers of analysis and measurements are ‘squeezed’ together into a giant “Bed of Procrustes” (really good read).
Sometimes it is easier to construct cryptic structures of numbers. It appears more prestigious. The problem arises when many of these numbers are merely masks for fuzzy ideas that are not clearly defined, particularly because of the air of ‘scientism’ and ‘exactness’ that this kind of message conveys.
Local and Culture and Creativity are important pillars that can add a lot to any National Innovation System. The game in essence is fine-grained and – to a degree – cultural.
Innovation is the more practical and market-oriented sibling of creativity, and to correctly understand it, a very different approach is needed. An approach that is clearer, more flexible and open, more local, and more contextual.
