The Hong Kong University Grants Committee’s Research Assessment Exercise 2020 proved the value of impact assessment and how it can build public trust in higher education, inform the funding process and change how excellence is defined
Research impact can be measured by the number of lives it has changed and how significantly those lives have been transformed, or a monetary value can be placed on the commercial success the research has found. What matters is that impact is measured at all; only then can universities present evidence that their research contributes to creating a better world.
This was the rationale behind the expansion of the University Grants Committee’s (UGC’s) Research Assessment Exercise (RAE) 2020 to include research impact. An ambitious assessment of the research impact of eight Hong Kong public universities, RAE 2020’s scope and methodology were inspired by research assessments of other jurisdictions, such as the UK’s Research Excellence Framework.
Participating universities submitted impact case studies that presented documentary evidence that their research made a difference beyond the campus. After the exercise, the impact case studies were turned over to analytics company Clarivate Analytics for analysis.
Chris Brink, convenor of the UGC’s RAE 2020, says the exercise reflects a perception shift in higher education. “The mission of higher education cannot only be to write papers to be read by other academics in academic terms,” Brink says. “It must also be in terms of the role we play in society, and there are all sorts of manifestations of this change in higher education. The impact agenda is one of them. But essentially, it is one way of responding to the question: what are universities good for in terms of the role that they play in society?”
Higher education has long been able to discuss excellence in terms of research output. How it communicates excellence through impact is a work in progress, Brink says, but the success of RAE 2020 bodes well for the future. It proved the quality of research from Hong Kong’s publicly funded universities. “I attended many of those panel meetings where the peer reviewers engaged with the submissions, both of the outputs and of the impact case studies, and all the environment statements,” Brink says. “Just from my own experience, I was very impressed by the quality of research, both of the outputs and the impacts, and of the research environment. I was impressed, but being impressed is not scientific evidence. Now we have the scientific evidence that says that the UGC can stand up on any platform, in any forum, and we can unequivocally state that the research conducted by the public universities of Hong Kong is of outstanding quality.”
The UGC usually undertakes such exercises every six years, but RAE 2020 was the first time that the UGC asked universities to submit impact case studies. The UGC scheduled a workshop to support the universities through the process, with professionals who worked on other similar exercises brought on board to share their experiences and coach universities through their submissions. In total, 345 impact case studies were submitted, and three were exempted from Clarivate’s report.
The exercise also assessed about 16,000 research outputs, involving 4,223 eligible academic staff and 191 research environment submissions. Brink says the results were pleasing but not surprising. A quarter of research submissions were graded as “world leading” and another 45 per cent as “internationally excellent”. Of the research environment submissions, 81 per cent were judged to be conducive to producing “world-leading” or “internationally excellent” research.
“Now we have this long and very thorough and very professional report that tells us that the impact was actually there,” Brink says. “The universities had been producing societal impact. It was a very good news message for the universities. It was also very good news message for the government and the taxpayer because the big question that gave rise to this in the UK and Hong Kong – and might I say in some other jurisdictions – is that if the government uses taxpayers’ money to invest in research, money that comes from the public, does society get any return on its investment? In Hong Kong, with this first exercise, there are hundreds of these examples, and we can say, ‘Here are evidenced reports of the impact on society. This is how society gets a return on its investment’.”
RAE 2020 will help inform funding decisions and it will build trust and prestige in Hong Kong’s higher education system. There are now examples of public money being invested in research and the investment paying off. No constraints were placed on universities as to what research impact they wanted to demonstrate. This was represented in a report that unearthed excellence in myriad sectors. A master’s student at one of the universities turned his research into DJI Drones, a company that posted revenues of $2.83 billion (£2.28 billion) in 2017.
An alternate wetting and drying irrigation technique allowed rice farmers in the Jiangsu province in mainland China to post an increase of CNY 4 billion (£466 million). The development of a non-invasive test for liver fibrosis saved the Hong Kong public health system HKD 13 million (£1.3 million). The report found that the societal impact of Hong Kong universities’ research supported the government’s strategic priorities and strengthened Hong Kong’s competitive advantage as a centre for international legal and dispute resolution services, and in international finance, transport and trade.
How we define research impact might be changing, but RAE 2020 proved that by measuring and documenting it we can build an incentive structure for universities to compete and collaborate on finding new ways to make a difference in society. “When we evaluate impact, we talk in terms of reach, in terms of people and audience, and also of significance, of the actual differences that such a research outcome has made,” says James Tang, secretary-general of the UGC. “You go through the 342 cases, and in all different ways they are making a difference to the lives of people here in Hong Kong and also elsewhere.”
Find out more about the University Grants Committee.
