Will you really hit the stars in the great REF in the sky?

There appears to be a deeply ingrained need in all humans to feel that, in however small a way, we have left our mark and made the world a better place. This is probably why, like many UK academics, I have spent many hours over the years chewing the fat with colleagues over the meaning of academic impact.

It is probably no surprise that there is little consensus on what it is and how it should be measured (if it should be measured at all). However, for most of us, our own assessment of our most significant contribution does not necessarily fix upon our publication with the highest metric. That paper might have just benefited unduly from fortunate timing or a large self-citing network. It might not boost our inner sense of self-worth as much as observers might assume.

Besides, there is far more to impact than purely academic impact. The UK’s 2014 research excellence framework recognised this, allocating 20 per cent of its marks to assessing case studies of impact beyond academia: on the “economy, society, culture, public policy or services, health, the environment or quality of life”.

The importance of this broader conception of impact has really been brought home to me in my new position as vice-chancellor of the Papua New Guinea University of Natural Resources and Environment. No doubt I am just adjusting my internal assessment criteria to justify my own existence but there seems to be a much greater opportunity here to help improve the lives of more people.

Thinking about how to measure value to society has ceased to be the navel-gazing exercise that it so often is in the UK. For instance, I have been involved in redrafting the funding model used to allocate resources to the four state universities within Papua New Guinea. When I arrived, the proposal was to fund research using a system based on the REF, and to measure academic excellence using similar criteria. Please let’s not get distracted here by unfair comparisons around how many of the region’s academics would make the REF cut in the UK. The point is that measuring academic value for the purpose of directing the investment of a nation’s wealth is a very important task – even more so when needs are so much greater and funds more limited. (Although, that said, it is routine here for universities not to receive the budget they are allocated, leaving them unable to pay salaries.)

In my view, Papua New Guinea would be prudent to direct funding towards research applied specifically to the country’s very real needs. Pure, theoretical research and the arts are wonderful things to support when your population is well-fed, well-educated and healthy. But we need to ensure those basics first.

We need to kick-start the virtuous circle of development, in which government money supports research, which drives forward development, which, in turn, generates more income for investment. Therefore, identifying what is worth rewarding really does matter.

Of course, you don’t have to go there to make a difference in a developing country. Many Western universities educate students from every corner of the globe. And lots of us are very familiar with the rewards and challenges of supervising postgraduates from very different educational systems. But perhaps we don’t always think through the implications.

There is a good case to be made for including the supervision of PhD students from developing countries high on your personal metric for “making a difference”. This becomes particularly evident when you visit their home institutions and discover just how many are run by academics with Western postgraduate qualifications. In fact, in many of the world’s universities – and governments – not having a higher degree from abroad is an insurmountable barrier to career progression.

In light of this, it is crucial for Western universities to consider what doctoral students, particularly those from developing countries, are actually going to do with their PhDs. My own research into the sex life of a common British weed was hardly the ideal training for running a university in Papua New Guinea, for example.

I rather suspect that some Western universities spotted this a while ago. But if they have, the time-lag is such that it hasn’t made much impact on the ground yet. So I would urge Western academics supervising postgraduates from developing countries to think about their likely career paths. Recognise that on returning home, they could rapidly find themselves catapulted into senior management roles, and consider helping them to develop their wider management skills.

If at the end of it all, there is a final REF in the sky to assess our overall impact, I would like to think that I will be scored highly for having been the postgraduate supervisor of both the future secretary to a prime minister and a future dean of science in developing countries. But it is a pity I did not recognise that at the time. If I had, I might have spent more time with those students talking about lab budgets – and less about which journal we should publish in.

John Warren is vice-chancellor of the Papua New Guinea University of Natural Resources and Environment.