Concern over pandemic’s unequal impact on TEF ratings

Experts have raised concerns about how the unequal impact of the pandemic on UK universities will be reflected in the results of the next teaching excellence framework.

Following the publication last month of the Westminster government’s response to the review of the assessment conducted by Dame Shirley Pearce, the Office for Students is expected to launch a consultation on changes to the exercise this spring, with the first round of results under the reformed model due to be announced in 2022.

Speaking at an event last month, Graeme Rosenberg, head of the TEF at the OfS, said the regulator expected submissions to the exercise to be made in late 2021.

“One of the issues we need to look at carefully is how the changes arising from the pandemic are reflected…but I’m not sure we can hold the TEF off until everything has settled down again, because it may not for some time,” Mr Rosenberg said.

Georgina Andrews, vice-provost of Bath Spa University, warned that the varying impact of the coronavirus crisis on different students, disciplines and institutions would make it difficult for TEF metrics to be assessed fairly.

For example, the National Student Survey is a part of the current TEF metrics, and students filling it out in 2021 will have had their second and third years heavily disrupted, Professor Andrews said. “I don’t think anyone would argue that it hasn’t been different to what students were expecting, however good the quality of the online experience is,” she told Times Higher Education.

The personal experiences of students – such as whether they have access to a quiet study space and good broadband service or whether they have additional financial worries because they lost their part-time job – have been uneven, Professor Andrews continued. This meant that universities with more privileged students might fare better than those that cater to more disadvantaged students.

There is also disparity between subjects, with some courses more easily adaptable to online learning than others. Some practical subjects have been allowed a degree of in-class teaching while others, such as drama and dance, are still largely denied that.

An increase in benchmarking in the TEF would be helpful, but overall, the best thing to do would be to delay the next round, Professor Andrews said.

David Phoenix, vice-chancellor of London South Bank University, agreed. “If TEF is to run in 2021, it is more important than ever that OfS uses data that is appropriately benchmarked and contextualised,” he said.

Professor Phoenix added that it would be “naive” to think that the severity of the pandemic’s impact would not vary by subject area, socio-economic background and mode of study, and that some students this year would also carry disadvantage from the pandemic into the careers market. Graduate employment is expected to continue to form a key part of the TEF.

Paul Ashwin, professor of higher education at Lancaster University, said the issue highlighted “one of the central weaknesses of the TEF. Rather than giving a sense of how the quality of education has improved over time, it simply provides a comparison of universities’ relative performance on the metrics at a particular moment in time,” he said.

It will be “very difficult, if not impossible”, to fairly accommodate differing levels of disruption in TEF ratings, Professor Ashwin warned.

The OfS said it was “considering how to account for the impact of the pandemic as we develop our consultation proposals. We expect to consult on specific timings as part of the overall consultation in spring.”

anna.mckie@timeshighereducation.com