Subject-level TEF gets frosty reception from the HE sector

“The first iteration of the teaching excellence framework has operated at institution level. But it is our belief that a subject-level TEF can provide even better information to students, and be an even more powerful driver of quality and value.”

So said universities and science minister Jo Johnson, addressing university leaders at the Reform thinktank this morning.

“I am yet to meet a vice-chancellor who is unaware of significant variations in quality between subjects and disciplines in his or her own institution,” he continued. “A subject-level TEF will empower them to make targeted interventions where they are most needed. Meanwhile students will be able to make better-informed decisions as they choose between courses and institutions.”

It sounds straightforward, right? And look – that helpful Mr Johnson has today also published the specifications for how the subject-level teaching excellence framework would work in practice. 

“We know that most students choose their subject first, and then choose between providers offering that subject,” this specification document states. “Students need to know how a provider’s teaching quality will relate to them in the subject that they are looking to study. The subject-level pilots will develop TEF so that it provides this for students.

Some 30-40 institutions are to take part in the pilot,  which will run from Autumn 2017 to Autumn 2018. 

Needless to say, this is a hugely complex thing to do – and it will potentially have huge workload implications for university staff. Mike Ratcliffe, a university administrator at the University of Oxford and regular online analyst of higher education, was not impressed.

Subject-level #TEF looks to exceed expectations for awfulness: 35 subjects, hypotheses x teaching intensity + statement = outcome. pic.twitter.com/Es6eCQCez3

— Mike Ratcliffe (@mike_rat) July 20, 2017

That's right. The pilot of the teaching level TEF will measure quality (or at least aim to measure quality) across 35 different subject areas. Performance will be measured by looking at the 10 existing TEF metrics plus a new one looking at “the teaching intensity a student experiences”. This is derived from data on contact hours, staff-student ratios and class sizes, gathered via institutional declarations and a student survey.

The implication for universities here is, on the face of it, quite simple: get your students in front of a member of staff more often, and you will perform better on this particular metric. The implication for staff is equally clear: spend more time with students. Professor Andrew McRae, head of English and professor at the University of Exeter, does not sound as if he’s looking forward to the extra bureaucracy.

Get used to ‘teaching intensity’: contact hours, class size, SSRs, placements and field work. #reformHEhttps://t.co/AbxhqzaDwH

— Andrew McRae (@McRaeAndrew) July 20, 2017

And there were plenty more dissenting voices too, including Pam Tatlow, chief executive of Million+...

Subject-level TEF a complex web which threatens to be a waste of student tuition fees - highly unlikely to promote innovation https://t.co/97lFOvF0na

— Pam Tatlow (@millionplusCEO) July 20, 2017

...and many more besides:

Subject-specific TEF pilot https://t.co/5UIUCgpaQA This edifice will eventually topple under the weight of its own complexity

— Tim Dracup (@TimDracup) July 20, 2017

More devts in the HE landscape - the subject level #TEF is coming. So more metrics scrutiny and submission writing on the horizon for us https://t.co/Sw1XY9WzG5

— Jo Norry (@JPNLeeds) July 20, 2017

TEF subject level pilots. This will be fun. Watch out for the GTQ (gross teaching quotient). Glad to be out of it https://t.co/QovmGWS0Dn

— Peter Starie (@stariep) July 20, 2017

By the way, if you were wondering how the subject-level TEF is going to work, two models are to be piloted. Model A is defined as the “by exception” model, and would give subjects the same rating as the university’s rating “where metrics performance is similar”, with a fuller assessment (and potentially different ratings) where metrics performance differs. Model B is defined as the “bottom-up” approach, and would mean fully assessing each subject to give subject-level ratings.

After a year of piloting, the Department for Education will evaluate the two models, with feedback on both the experience of those taking part and and the costs of participating to be assesed. In addition, there will be a consultation launched later this year, allowing universities, students, employers and other interested parties to comment on the proposed design. 

The government also plans to commission research to test aspects of subject-level TEF with “a wide group of students”, with findings from the pilots, the consultation and the student research feeding into the final design. It is hoped that actual assessments will begin in academic year 2019/20, with the first subject-level ratings published in spring 2020.

Regardless of which approach is taken, one thing is for sure: it is happening. Mr Johnson closed by saying the government’s “focus as we implement the Higher Education and Research Act” will include delivering “the Teaching Excellence Framework at subject level”.

“Students applying to university this year, and across this parliament, will have more information, more choice and more flexible ways of learning than ever before,” he concluded.