Impacts don't just happen

The impact of mathematics in every walk of life is astounding. In a 2013 report commissioned by the Engineering and Physical Sciences Research Council, Deloitte estimates that it directly accounts for 10 per cent of all jobs and 16 per cent of total gross domestic product in the UK. Yet mathematics scores less well than other sciences on standard measures of knowledge exchange, such as patents and contract research – and this is a concern to administrators in an increasingly metrics-conscious world.

Mathematicians have long argued that the impact of their research is long term, hard to predict and often happens via multifaceted interdisciplinary work. The literature on knowledge transfer recognises many kinds of impacts. There are direct and relatively tangible “instrumental impacts”, resulting in concrete applications. There is “attitude or cultural change”. There is “capacity building” through education and training. There are “conceptual impacts” that can reshape disciplines, policies or industries (think of Karen Spärck Jones showing in the 1960s that statistics, not formal grammar, was the key to computers understanding language, leading to tools such as Google Translate). And there is the creation of “enduring connectivity” through long-term relationships with research users.

A study we carried out in the wake of the UK’s 2014 research excellence framework and published in January under the title “Slightly dirty maths: The richly textured mechanisms of impact” found that mathematics research generated a full range of impacts – particularly conceptual impacts. However, our analysis of the 209 case studies submitted to the REF by maths departments showed a strong emphasis on linear narratives, concentrating on instrumental impacts.

In interviews, department heads confirmed to us that these are what they thought the REF panels would want. Yet the case studies also cast incidental light on a complex ecosystem of research, researchers and research users, with multiple interdependencies between impacts. We saw many impacts arising out of strong, often informal, long-term relationships. Users learn what mathematicians are doing, and mathematicians learn what might be useful to users. For instance, an important role is played by mathematically informed “knowledge intermediaries”. These individuals and organisational entities (such as Innovate UK’s knowledge transfer network in industrial mathematics) build often undervalued but vital bridges between academic mathematicians and users of research.

Ironically, the more complex the impact ecosystem, the easier it is likely to be to extract tidy linear narratives – but the less representative of the true picture such narratives will be. Our interviewees perceived interdisciplinarity as perhaps particularly important for impact generation by mathematics (think of algorithms used in engineering, epidemiology or transport) but expressed a lack of confidence in how such impacts might be treated by REF panels.

This is not an issue for mathematics alone. Although the definitions of impact adopted by the REF were relatively broad across the board (albeit more so in some disciplines than in others), it seems that not only departments but university research leaders “played it safe” and focused on instrumental impacts.

And although the form of the next REF is not yet finalised, universities are already collecting and prioritising draft impact case studies. The real, perverse risk is that impact assessment will retard rather than encourage impact generation if these artificially constrained definitions – or perceived definitions – of impact become set in stone. It could distort collective understanding of impacts, and of how they come to be, encouraging a false perception that a linear causality connects academic papers with instrumental impacts. This, in turn, could shape actions taken (or not taken) by researchers, departments and universities.

So we would urge UK university departments to value all impact types and to accept that they develop over time, within complex ecosystems of formal and informal relationships that may well be interdisciplinary. And if they want to do well in future REFs, departments must also take steps to facilitate the creation and maintenance of such relationships – including by the provision of incentives and rewards for those who pursue them.

Linear narratives of impacts unfolding over time are useful to track progress. But they need to be seen as a part of diverse portfolios of truly rich stories, celebrating the human and interactive nature of impact generation.

Laura Meagher is senior partner at Technology Development Group, a strategic change consultancy in Fife, Scotland. Ursula Martin is professor of computer science at the University of Oxford.