Guidance suggests TEF is now about government priorities, not teaching

In many ways, apart from a change in name, little has changed from year two of the teaching excellence framework (TEF) in the guidance for year three set out in the Teaching Excellence and Student Outcomes Framework (TESOF) Specification earlier this month.

The criteria and the guidance are basically the same, with more explicit information on how to deal with things such as indicators with very high or low values, split metrics and the metrics of institutions that have a majority of part-time students.

However, while the changes are rather small, they show how the TEF (the acronym used by the government has stayed the same despite the addition of the "S" and the "O") is moving from a reasonable, if limited, measure of teaching quality to a device by which ministers can send sharp messages to universities about where they should focus their attention. The temptation to create a set of metrics that reflects government priorities rather than teaching quality was always a danger in the use of a centrally devised set of metrics. This danger is now beginning to be realised.

The first message being sent is that levels of graduate employment is now the most important measure of the teaching excellence of a university. This is because the three metrics based on what students think of their courses from the National Student Survey have each been downgraded to half a metric (or a "met").

To reinforce further the importance of employment, there are two new supplementary metrics based on the longitudinal education outcomes data (hint: outcome here means employment).  One is the proportion of students earning more than the median salary (which we are breathlessly informed is below that of nurses and so does not merely celebrate universities who provide the kind of financial whizz kids capable of bringing the economy to its knees) and the other being engaged in sustained paid employment or study after the three years.

That it feels anachronistic to point out that employment rates are not in the direct control of universities and are related more strongly to factors such as institutional reputation, location and subject mix than teaching quality shows how far the TEF is straying from any evidence-informed construction of its criterion for assessing teaching excellence.     

The second message, sent through the other new supplementary metric on grade inflation, is that in universities with excellent teaching, only a defined proportion of students can ever do well. Two "legitimate" reasons for a university having higher grades awarded over time are identified: taking in students with higher grades or “clear and compelling evidence that the absolute standard of assessments at that provider have substantially increased in objective difficulty over that period”.

The first problem with this is that, while it sounds very rigorous to write in terms of "the absolute standard" and "objective difficulty", years of research into assessment show that such ideas are a chimera because, at its heart, the rigorous assessment of students’ understanding of complex knowledge is about human, fallible, judgement.

The second problem is that it implies that unless degree programmes get "substantially" more difficult over time or take in students with higher entry grades then any increases in degree class awarded are because of grade inflation. Under this mind-blowing way of approaching teaching excellence, rather than wasting their time carefully designing curricula that invite students into a productive conversation with disciplinary and professional knowledge, what excellent teaching should do is simply grade against a normal distribution curve.  

Added to this impoverished notion of what high-quality teaching and assessment might be and despite its commitment to not interfere with institutional autonomy, these assertions also put the Department for Education in the position of adjudicating legitimate reasons for increases in student performance over time. Despite this already impressive workload for a document of this kind, the framework specification also finds time, in passing, to write off the external examiner system entirely by asserting that the degree classifications awarded to students are “determined entirely by the provider”.

Taken as a whole, these changes tell us that, under the TEF, universities with excellent teaching are those that design programmes that ensure only a fixed proportion of students can do well and which get the most students into jobs. They also hint that doing well in the TEF is becoming less about providing a high-quality education and more about addressing the government’s priorities for undergraduate degrees.

If it continues is this way, a gold award will be more of a sign of being "government approved" than telling students anything useful about teaching quality at a particular university.

Paul Ashwin is professor of higher education at Lancaster University and a researcher in the ESRC-HEFCE funded Centre for Global Higher Education.