Changes to England’s teaching excellence framework will “further undermine” its validity and shift it towards confirming existing hierarchies, it has been warned.
Significant changes to the TEF – including halving the value of National Student Survey results in core metrics and introducing new data on graduate earnings in a “supplementary metric” – were announced by Jo Johnson, the universities and science minister, in a speech at the Universities UK conference on 7 September. The changes, which would be in effect for TEF assessments in 2017-18 and 2018-19, were detailed in a subsequent Department for Education policy document on “lessons learned” from the last exercise.
After the independent review of the TEF in 2018, and subject to the findings of that review, the TEF will move to a five-year cycle, the DfE document also announces. A five-yearly TEF “will deliver value for money”, it adds.
The weight of each NSS metric, looking at students' views on teaching, “will be halved for the purposes of determining the initial hypothesis”, the paper says.
Meanwhile, a new supplementary metric on each institution’s graduate employment outcomes – one that will not affect the initial hypothesis but will be considered alongside provider submissions – will be created using the DfE’s Longitudinal Education Outcomes data. There will continue to be a core metric derived from the Destinations of Leavers from Higher Education survey, into which LEO will be incorporated by 2020.
There will be two LEO metrics: one on the proportion of an institution’s graduates in “sustained employment” or further study three years after graduation, and another on the proportion of graduates in sustained employment “earning over the median salary for 25-29 year olds”.
The median salary figure is currently £21,000, the DfE paper notes. This salary point against which universities will be judged also happens to be the repayment threshold on student loans.
LEO metrics will be benchmarked “in a similar way to core metrics”, the paper says, without making any mention of whether benchmarking will take account of regional earnings variations.
Another supplementary metric on grade inflation considered alongside provider submissions will also be introduced, which will “record the proportion of firsts, 2:1s and other grades as a percentage of all classified degrees at that provider one, two, three and 10 years before the year of assessment”. Mr Johnson warned in his speech that grade inflation was “ripping through English higher education”.
Pam Tatlow, chief executive of MillionPlus, said of the grade inflation metric: “There are risks to institutional autonomy as a result of his [Mr Johnson’s] approach that universities will want to consider carefully.”
One factor behind reducing the NSS weighting could be anxiety over the potential impact of any continuing NSS boycotts by students’ unions – some of which are angry about the TEF being used to raise fees in line with inflation.
Reducing the weight of NSS scores – where many Russell Group universities traditionally fare poorly – and introducing additional graduate earnings metrics – where more prestigious institutions that attract wealthier students perform well – could have a major impact on the TEF. The exercise may be shifted towards “confirming sector hierarchies”, said Gordon McKenzie, chief executive of GuildHE.
Paul Ashwin, professor of higher education at Lancaster University, said the introduction of graduate earnings data as a supplementary metrics “is problematic because…it has no established relationship to the quality of teaching. Graduate earnings are related to university reputation, location and subject-mix rather than the quality of teaching. They are also not under the direct control of universities because they are shaped by the state of the economy.”
The changes “will further undermine the validity of the TEF”, he added.
Professor Ashwin continued: “One predictable outcome of the shift to the greater use of graduate earnings will be to boost the position of Russell Group universities in the TEF.”
