England’s investment of more than £4 million in studying learning gain could be wasted after much-heralded research projects drew to a close without a clear way forward.
The now-defunct Higher Education Funding Council for England launched 13 pilot projects at more than 70 institutions in 2015, with the aim of finding ways to measure the improvement in students’ skills and competencies over the course of their university study.
At the time, it was hoped that the trials – which also included a bid to experiment with national standardised tests – could point to a way of comparing institutional performance that could be incorporated as a new metric into an assessment such as the teaching excellence framework.
However, England’s new higher education regulator, the Office for Students, quietly published the evaluations of the learning gain work last month and said that “no further research” was planned.
The research concludes that there was “no silver bullet” that would enable a regulator to “accurately and effectively measure student learning comparatively across subjects of study and institutional types, despite appetite from government ministries and the media”.
The Hefce-commissioned pilots examined nearly 30 approaches to measuring learning gain, exploring areas including cognitive development, “soft skills” and employability.
The standardised testing initiative – the National Mixed Methodology Learning Gain project – was scrapped last year after not enough students could be persuaded to participate.
Camille Kandiko Howson, associate professor of education at Imperial College London and author of the main evaluation report, said the pilots had shown the limitations of measuring learning gain.
“It would be appropriate to measure within small subject groups or, possibly, across specific institutional types, but not broad metrics that would be relevant for all students, all subjects across all institutions,” Dr Kandiko Howson said.
However, because the OfS regulates the sector through the lens of the institution, “there is a mismatch when it comes to measuring learning gain and the way the OfS wants to use data”, she continued.
“A lot of this is shift from Hefce as a funding council to OfS as a regulator…Hefce’s remit included looking into teaching and learning enhancement and providing data to help institutions improve,” she said. “In terms of taking the work forward, the OfS put this activity in its structure under teaching and learning, and they have no budget for this.”
Dr Kandiko Howson pleaded for this not to be the end of major learning gain research in the UK. “Perhaps a body like Universities UK, a learned society or even Advance HE could pick it up,” she said.
Carol Evans, professor of higher education at the University of Southampton, agreed.
While acknowledging that there had been problems around the lack of planning before the launch of the pilots and with the widespread use of surveys, the primary flaw was in chasing national comparability, she explained, saying it was “fool’s gold, it won’t help us”.
“However, if you look at the micro level, there are many possibilities. Learning gain is nuanced by the context – you have to look at the students,” Professor Evans continued. “Learning gain has massive potential if we use it for the process, not the outcomes.”
In particular, learning gain data could be used to help a lecturer enhance their teaching, Professor Evans said. “If the government is serious about investing in enhancing teaching and learning, this is really valuable, but where will the funding come from?” she asked.
An OfS spokesman said the regulator was “currently taking stock of the findings from the evaluations of each strand of the learning gain programme. Currently, no further research into learning gain is planned.”
