Mock TEF results revealed: a new hierarchy emerges

Our data team’s modelling of the potential teaching excellence framework results shows the Russell Group losing out to a new elite

June 23, 2016
Chris Havergal
University of Oxford Bodleian Library viewed from ground
Source: Getty

The results of the 2017 TEF are now available

Asked to list the elite institutions of English higher education, a few names would easily roll off most tongues, all of them members of the Russell Group. One reason for this is their age. Another is their high entry standards. But, especially when it comes to league tables, their pre-eminent status is largely due to the strength and scale of their research.

But how long research will continue to be the primary driver of prestige, in England at least, is open to question. The government is making a concerted push for a sector with – as a previous White Paper put it – students at the heart of the system. Jo Johnson, minister for universities and science, argues that some institutions have traded on their world-class research reputation without giving their £9,000 fee-paying students an educational experience to match. Enter the teaching excellence framework (TEF). This is the government’s attempt to assess higher education providers on their educational standards, to give students better information about what they can expect from their course and to give universities a financial and reputational incentive to pull their teaching socks up.

The big question is how much the TEF will shake up the established hierarchy. How closely will the results resemble the research excellence framework, or the world university rankings? According to exclusive research by Times Higher Education, the answer is likely to be: not very closely at all.

THE’s data team modelled the potential results of the TEF for 120 UK universities using the three supposed core aspects of teaching performance that the government recently confirmed it will measure in the early stages of the TEF: graduate employment, student retention and student satisfaction.

Times Higher Education mock TEF: top 10

Benchmarked TEF rank Absolute TEF rank University name Absolute score Benchmarked score
1 5 Loughborough University 92.8 100.0
2 13 Aston University 84.8 97.9
3 45 De Montfort University 64.1 93.9
4 15 Swansea University 83.6 93.7
5 =22 University of Kent 77.5 91.8
6 34 Coventry University 70.2 91.4
7 =10 Keele University 87.3 89.3
8 12 University of Surrey 86.4 86.3
9 =2 University of Bath 93.3 84.7
10 7 Lancaster University 88.9 84.1

Scroll down to view the top 120 results of the Times Higher Education mock TEF
Download the full results of the mock TEF in PDF format

Looking at institutions’ raw scores on those metrics, the traditional hierarchy is still evident, with the Russell Group and some other select pre-92 universities leading the way. But that dominance seems to be based, in many cases, on the types of students that these institutions recruit. When the scores are adjusted for factors such as subject mix, entry qualifications and ethnicity – as they will be in the real TEF – a very different picture emerges.

Far from the “golden triangle” of Oxbridge and London reigning supreme, it is a Midlands triangle of Loughborough, Aston and De Montfort universities that emerge as the top performers. Coventry University is another post-92 institution that figures very highly, and also conspicuous in the top 10 are a number of smaller research-intensives, such as the universities of Swansea, Kent, Surrey, Bath and Lancaster.

Some Russell Group institutions still do well. The University of Cambridge, top on the absolute score, is 12th after benchmarking, while Durham University is 16th. The University of Oxford, fourth on the absolute ranking, is a little way behind, coming in 28th – seven places below its near neighbour Oxford Brookes University. Other Russell Group institutions do considerably less well: the University of Edinburgh comes 89th, for example, while the University of Bristol comes 87th and the University of Liverpool 80th.

Phil Baty, THE’s rankings editor, says that the TEF “could send shockwaves not just through the UK but across the world”.

Universities’ absolute and benchmarked scores in mock TEF

Universities’ absolute and benchmarked scores on mock TEF (23 June 2016)

“Some household names – major international university brands – previously judged to be world-class based mainly on their proven research excellence, will suddenly be exposed to the world as failing to deliver the expected results for their students – surely their most important stakeholders,” Baty says. “The metrics used may continue to be highly controversial, and will continue to be debated, but they will bear the powerful stamp of government approval, so they will be taken seriously as a reference point around the world.

“Of course, the prestige that comes from a long history of research excellence, membership of the elite Russell Group and selective entry standards will continue to attract students internationally, but there is huge potential for disruption.”

Baty’s point about the controversy over the metrics used is an important one. There are many who would argue that the best university teaching involves making students feel challenged and even uncomfortable; something that cannot always be associated with satisfaction, as measured by the National Student Survey. Likewise, many are concerned that assessing universities on graduate employment, via the Destinations of Leavers from Higher Education survey, ignores so much of the personal development that is central to higher education.

It is also worth noting that THE’s modelling of the TEF metrics cannot take into account the additional institutional evidence that – as set out in last month’s White Paper, Success as a Knowledge Economy – will be submitted by each provider to special TEF panels, and will be used to supplement the core data before a final verdict is reached.

But many of the universities that perform best in our mock exercise will take the results as confirmation of what they have been saying for a long time: that smaller, campus-based, research-active institutions offer the best student experience. Helen Higson, Aston’s deputy vice-chancellor, argues that if the TEF metrics can be got right and put in the appropriate context, it will be positive for the sector to have greater focus on teaching standards.

High and low scorers for graduate destinations

Absolute

Rank Name Score
1 Imperial College London 91
2 University of Cambridge 89
=3 University of Birmingham 85
=3 King’s College London 85
=3 University of Bath 85
6 University of Sussex 84
=7 Loughborough University 83
=7 Durham University 83
=7 University College London 83
10 Lancaster University 82
     
=109 Goldsmiths, University of London 55
=109 University of Gloucestershire 55
=109 Bath Spa University 55
=109 University Campus Suffolk 55
=109 University of Westminster 55
=114 Southampton Solent University 54
=114 University of Wolverhampton 54
116 Liverpool Hope University 53
117 Newman University 52
118 University of Wales Trinity Saint David 50
119 London Metropolitan University 47
120 University of East London 45

Benchmarked

Rank Name Distance from benchmark
1 De Montfort University 14
2 University of Sussex 13
=3 University of Cambridge 10
=3 University of Kent 10
=3 Robert Gordon University 10
=6 Swansea University 9
=6 Coventry University 9
=6 University of Huddersfield 9
=6 University of Birmingham 9
=6 Lancaster University 9
=6 Loughborough University 9
     
=111 Plymouth University –9
=111 Liverpool Hope University –9
=111 Teesside University –9
=111 Ulster University –9
=111 Newman University –9
116 University Campus Suffolk –10
117 London Metropolitan University –11
118 University of Wales Trinity Saint David –12
119 University of East London –13
120 Queen Margaret University –16

“I went to a very research-intensive university where I wasn’t taught by anyone who knew how to teach or why they were teaching me. Although it was great to be taught by the top minds, they weren’t necessarily the top teachers,” says Higson. “There is still a lot of public resource going into supporting students, so I think to run any [course] that isn’t excellent – that is run by ‘amateurs’ – is ethically not right, especially as a lot of individuals are now putting a lot of money into their education.”

Examination of the top performers’ results reveals that while they tend to perform above their benchmarks on student satisfaction and retention, it is often on graduate employment where they really excel, coming in well above what might be expected.

The top-performing Russell Group institutions – which also include the universities of Birmingham and Exeter – tend to perform well beyond expectations on graduate employment, but are closer to their benchmarks on the other metrics. Many other members of the mission group struggle to significantly outperform their benchmarks, and some fall well short in some areas.

All this will be useful and fascinating information for academics and university administrators, but the TEF is likely to have the significant impact that the government desires only if students take notice of it when choosing where to study, and if employers do the same when deciding which graduates to recruit. For instance, Russell Group institutions attract a significant proportion of international students, and the fee revenue that comes with them (see 'Market share of non-UK undergraduates' graph, below), so any dent to their reputation could lead to a major reallocation of resources within the sector.

As Baty says, the backing of the government has the potential to give the TEF significant weight, and this is welcomed by Robert Allison, Loughborough’s vice-chancellor. Like many university leaders, he stresses the importance of making sure the TEF metrics truly reflect excellence, but he believes that the TEF has the potential to “challenge orthodoxies” in the sector.

High and low scorers for degree completion

Absolute

Rank Name Score
1 University of Cambridge 99
=2 London School of Economics and Political Science 98
=2 University of Oxford 98
=4 University of Bath 96
=4 University of Exeter 96
=4 Imperial College London 96
=4 University College London 96
=4 Durham University 96
=4 University of St Andrews 96
=10 University of Bristol 95
=10 Loughborough University 95
=10 University of Warwick 95
=10 Newcastle University 95
=10 University of Birmingham 95
     
=111 University of Bedfordshire 76
=111 University Campus Suffolk 76
=113 University of West London 75
=113 Middlesex University 75
115 University of Wolverhampton 74
=116 University of East London 73
=116 London South Bank University 73
118 Glyndwr University 72
=119 University of Bolton 71
=119 London Metropolitan University 71

Benchmarked

Rank Name Distance from benchmark
=1 Aston University 6
=1 University of Chichester 6
3 Bishop Grosseteste University 5
=4 Loughborough University 4
=4 Oxford Brookes University 4
=4 University of Lincoln 4
=4 University of Kent 4
=4 Swansea University 4
=4 University of Northampton 4
=4 Birmingham City University 4
     
=109 University of Stirling –5
=109 Soas, University of London –5
=109 Middlesex University –5
=109 University of Central Lancashire –5
=109 University of Aberdeen –5
=114 University of Wolverhampton –6
=114 University of Bolton –6
=114 University of the West of Scotland –6
=114 Robert Gordon University –6
=118 Queen Margaret University –8
=118 London Metropolitan University –8
120 Abertay University –9

“I’m in no doubt that in the TEF there will be some of the Russell Group right at the top and I won’t be surprised if there are some quite a long way down,” says Allison. “Will the Russell Group implode as a consequence? I doubt it. But will that challenge the message that the Russell Group gives out that ‘we are the country’s elite universities’? I hope it does, because if there is evidence to suggest constructs like that are true, I’m pleased it reinforces it. But if there is evidence that indicates constructs like that aren’t necessarily everything they are made out to be, that would be an interesting consequence.”

An institution’s performance in the TEF will be communicated to students and employers via the rating they will be awarded. They will be judged to “meet expectations”, or to be “excellent” or “outstanding”. The government has already signalled that it expects about 20 per cent of institutions to fall into the first, lowest category, with between 50 and 60 per cent of institutions deemed excellent and the other 20 to 30 per cent outstanding.

Based on THE’s analysis, this entails that no Russell Group universities would be judged merely to “meet expectations”. But as few as five of the 24 members – add Newcastle University to the four previously mentioned – could be judged “outstanding”, with the universities of Leeds and Oxford on the borderline if the government opts for the lower cut-off. Cardiff University will also be in a similar position if Wales decides to participate in the full TEF. (The devolved nations have not yet decided whether to do so.)

In England at least, the strongest public indication of an institution’s performance is likely to be the tuition fees it is allowed to charge, with excellent and outstanding institutions permitted to make increases in line with inflation for the 2019-20 academic year onwards, and those that meet expectations permitted to raise their fees at only half that rate. However, this may limit the transformative power of the TEF since the worst-performing institutions in relative terms tend to be among those that do not do well on an absolute ranking or other measures of excellence commonly used in the sector.

High and low scorers for student satisfaction

Absolute

Rank Name Score
1 Keele University 95
=2 University of Surrey 92
=2 University of Winchester 92
=2 University of Essex 92
=2 University of East Anglia 92
=6 Loughborough University 91
=6 Swansea University 91
=6 Coventry University 91
=6 Lancaster University 91
=6 Newcastle University 91
=6 University of Oxford 91
=6 Bangor University 91
     
=111 University of Westminster 80
=111 Glyndwr University 80
=113 Bournemouth University 79
=113 University of South Wales 79
=113 University of West London 79
=113 University of Wales Trinity Saint David 79
=113 London Metropolitan University 79
=118 University of Cumbria 78
=118 University of St Mark and St John 78
=118 University of East London 78

Benchmarked

Rank Name Distance from benchmark
1 Keele University 8
2 University of Winchester 7
=3 Loughborough University 6
=3 Coventry University 6
=3 University of Surrey 6
=3 University of Essex 6
=3 Bath Spa University 6
=8 University of East Anglia 5
=8 Bangor University 5
     
=109 Bournemouth University –5
=109 University of South Wales –5
=109 University of Westminster –5
=109 Glyndwr University –5
=109 University of Wales Trinity Saint David –5
=109 London Metropolitan University –5
=115 London School of Economics and Political Science –6
=115 King’s College London –6
=115 University of Cumbria –6
=115 University of West London –6
=119 University of St Mark and St John –7
=119 University of East London –7

Other significant brakes on the TEF as an agent for change are likely to be the enduring power of prestige and reputation and the lack of assessment at subject level until the fourth year of the exercise. This lack of disciplinary breakdown may prove to be particularly damaging to the government’s ambitions for the TEF to feed directly into sixth-formers’ choice of institution given that most choose course before institution, and there is likely to be widespread variation in standards of teaching on the various courses across institutions.

Stephen Isherwood, chief executive of the Association of Graduate Recruiters, warns that institutional-level data are likely to be “too generic” to be used in student application or post-university hiring decisions. Ultimately, there is also a question about how important relative TEF scores will be to employers – and, consequently, to applicants – when the selectivity of elite institutions is likely to ensure that their graduates will still have among the strongest outcomes in absolute terms.

Nick Hillman, director of the Higher Education Policy Institute, started to explore ways of assessing providers on their teaching standards while he was a special adviser to David Willetts, the former universities and science minister. He argues that the TEF will “quite quickly become a significant source of information used by applicants”. But even he accepts that it will take time to change employers’ views of universities, and that there will probably “always be a cachet from being really old or from being really research-intensive”.

The Russell Group, for its part, cautions that it would be “wrong to attach any weight” to the THE exercise, since it is based on only one year’s data, rather than the government’s preferred three, and because, according to the mission group, the methodology “doesn’t meet Office for National Statistics standards”.

Market share of non-UK undergraduates

Market share of non-UK undergraduates (23 June 2016)

Note: Universities are ordered according to their benchmarked TEF scores, with the highest scorers towards the left. In a small number of cases, data are not shown, either because the data are not collected (Northern Ireland) or because they were withheld by the institution.

Duncan Ross, data and analytics director at THE, responds: “Despite not being a public body (and therefore being outside the scope of the legislation), THE, and this analysis in particular, meets all of the relevant principles of the UK Statistics Authority Code of Practice for Official Statistics. This analysis is clearly in the public interest, as it gives a new and vital insight into the effectiveness of teaching in UK institutions.”

Wendy Piatt, the Russell Group’s director general, acknowledges that there is “always room for improvement” in teaching, but argues that “robust and credible measures of teaching quality will take time to develop through piloting and review. A huge amount of time, effort and resources have been devoted to improving the education and student experience our universities provide. This is reflected in feedback from employers and our students who, year-on-year, express above-average levels of overall satisfaction with the quality of their course.”

Beyond shaking up the traditional hierarchy of institutions, our mock TEF also paints a worrying picture for London. Belowbenchmark NSS scores drag down University College London, King’s College London and the London School of Economics: all members of the Russell Group. There are no London institutions in the “outstanding” category, while one in four of those in the “meets expectations” group is from the capital, with retention and graduate employment often proving problematic too.

London institutions currently attract disproportionate numbers of international students, but Hillman says that the TEF could encourage them in particular to look further afield: “International students will have to use data because it is hard for them to travel to London before they apply,” he notes.

If the devolved UK governments do take part in the TEF, the scores could also give some indication of the relative performance of institutions in different parts of the UK, and the impact of different funding systems. While the average score achieved in relative terms across the UK is 55.8, Scottish institutions – which have not benefited from £9,000 tuition fee income from domestic students – average just 44 points. Wales and Northern Ireland, which both face significant cuts in government funding for higher education, average 49.3 and 55.1 respectively.

Whatever happens – and whatever view people take of what the core metrics actually measure – our exercise suggests that the TEF will enable us to see one of the most renowned university systems in the world from a new and potentially enlightening angle.

Methodology for the THE mock teaching excellence framework

Times Higher Education’s modelling of the teaching excellence framework draws on the government’s core measures for the opening stages of the exercise: student satisfaction, student retention and graduate employment.

On satisfaction, THE’s data team uses the results of the 2015 National Student Survey. Specifically, we use question 22, relating to overall satisfaction, benchmarked for subject and students’ age, ethnicity, gender and disability. The government has recently announced that it plans to use the results of specific NSS questions on assessment and feedback, teaching and academic support, but benchmarks for these are not publicly available.

On student retention, we use the Higher Education Statistics Agency’s performance indicator T5 for 2013-14, which measures the proportion of full-time first degree students who are on course to achieve their chosen qualification. This is benchmarked for subject and students’ entry qualifications and age.

The government has recently said that it plans to draw on a different performance indicator, T3, which measures the proportion of students that continue with their studies beyond their first year. However, the two measures are strongly correlated, and it could be argued that T5 is a better measure of teaching quality because it does not reward universities where large numbers of degree students leave with diplomas instead.

To measure employment, THE commissioned a graduate destination marker from Hesa, which mimics what the government is consulting on for the TEF, measuring the proportion of university leavers who go into professional-level jobs, as defined by the Office for National Statistics, or postgraduate-level study. We also commissioned benchmarks for each institution, based on subject mix and students’ entry qualifications, age, ethnicity and gender.

Data for 120 institutions from across the UK are included in the final table. Some small, specialist providers have been excluded, either because they have potentially anomalous results in areas such as graduate employment or because they have limited numbers of full-time undergraduates.

Giving each of the three metrics equal weighting, an absolute score for each institution is generated, as well as a relative score reflecting benchmarking.

The THE exercise is unable to draw on qualitative submissions from institutions, which will be used in the real TEF to determine final ratings. Another caveat is that THE uses only one year of data, rather than the three that are planned for the TEF. However, Nicki Horseman, lead higher education data analyst for THE, doubts that using three years instead would affect the results significantly: “Using three years is aimed at providing sufficient data to break down the student body into smaller cohorts, such as part-time students. Here, we are not attempting that finer-grained analysis,” she says.

Horseman says the process of modelling the TEF highlights significant issues that the government will face when deciding how to rate providers: “Some institutions are strong across the board on all three measures, but it’s much more complicated when an institution is strong on one and weak on another,” she says. “How is that actually going to be assessed?”

Horseman says that another challenge will be how to divide universities into the government’s three categories – “meets expectations”, “excellent” and “outstanding” – when three clear groupings are not evident in the metrics. Instead, universities’ performance appears to range along a continuum, with a few outliers at either end.

“One of the criticisms of rankings and league tables is that we have very small differences between individual places,” says Horseman. “Institutions can get overly concerned about ranks when there is actually little difference in scores, but the TEF is proposing three assessment levels. Everybody wants to be outstanding and I don’t know how there will be evidence of a real difference between them.”

TEF v REF: universities' scores on the 2014 REF and our mock TEF benchmarked scores

Universities' scores on the 2014 REF and mock TEF benchmarked scores (23 June 2016)

The advantages that large pre-1992 institutions enjoy in research are unlikely to give them a similar head start in the teaching excellence framework, Times Higher Education’s analysis suggests.

Here, we plot the data from our modelling of the core TEF metrics against institutions’ intensity-weighted grade point average in the 2014 research excellence framework, calculated by multiplying the standard GPA by the proportion of eligible staff submitted (“REF 2014 rerun: who are the 'game players'?”, Features, 1 January 2015). Dot sizes correspond to each provider’s student numbers.

Members of the Russell Group cluster at the top end of the REF scale. These institutions also tend to perform well on the TEF measures, but are not among the top tier after benchmarking.

The graph shows that, instead, it is often significantly smaller institutions that perform best on the core TEF measures of retention, student satisfaction and graduate employment. While many of the top performers on these metrics also have strong research profiles, they are not among the top research performers, and some of them had quite low scores for research power: a hybrid measure that mimics the research funding formula by combining quality and volume.

The lower performers in REF terms – most of which are post-1992 institutions – display a greater diversity of scores on the TEF metrics.

Nicki Horseman, lead higher education data analyst for THE, says the results reveal only a weak correlation between REF and TEF performance.

“Some of the institutions which are doing best on the measures we put together are smaller, campus-based universities,” she said. “It may be that they have a stronger connection with their students and more evidence of a community.”

How well do universities perform in getting their graduates into employment?

Getting students into a job is unlikely to be enough to guarantee success for a university in the teaching excellence framework; placing them in what the government calls “graduate-level employment” is likely to be key.

The government is exploring whether it should introduce a metric for the proportion of graduates in highly skilled employment, as defined by the Office for National Statistics: an approach followed by Times Higher Education in its modelling of the possible TEF results.

This graph compares universities’ performance on the proportion of their graduates in all employment six months after graduation with THE’s measure of the proportion of students in professional employment and postgraduate study after six months. After benchmarking for the type of students it recruits and courses it runs, De Montfort University stands out as the exceptional performer on both measures. Other top performers include the universities of Sussex, Hertfordshire and Huddersfield, plus Coventry, Lancaster and Robert Gordon universities.

But, interestingly, there are also a number of providers that are very successful at getting their leavers into work but are not as good at placing them in what are deemed professional jobs. Institutions in this group include the universities of West London, Derby, Wolverhampton and Northampton.

On the flip side, two Russell Group members – the London School of Economics and the University of Glasgow – stand out for being comparatively poor at getting their students into employment within six months. However, when these students do get jobs, they are more likely to get a professional role.

What might the teaching excellence framework results look like?

Benchmarked TEF rank Absolute TEF rank University name Absolute score Benchmarked score
1 5 Loughborough University 92.8 100.0
2 13 Aston University 84.8 97.9
3 45 De Montfort University 64.1 93.9
4 15 Swansea University 83.6 93.7
5 =22 University of Kent 77.5 91.8
6 34 Coventry University 70.2 91.4
7 =10 Keele University 87.3 89.3
8 12 University of Surrey 86.4 86.3
9 =2 University of Bath 93.3 84.7
10 7 Lancaster University 88.9 84.1
11 =42 University of Winchester 66.7 82.7
12 1 University of Cambridge 100.0 82.5
13 =37 University of Essex 68.9 81.9
14 54 University of Lincoln 59.5 80.6
15 20 University of Sussex 78.5 79.3
16 6 Durham University 91.1 79.1
17 53 University of Huddersfield 59.7 79.0
18 =10 University of Birmingham 87.3 78.7
19 50 University of Portsmouth 61.0 76.6
=20 62 University of Chichester 54.5 76.2
=20 33 Oxford Brookes University 70.7 76.2
22 8 University of Exeter 88.1 76.0
23 19 University of East Anglia 79.4 74.7
24 9 Newcastle University 87.8 73.7
25 14 University of Leeds 83.7 73.0
26 56 Nottingham Trent University 58.6 72.2
27 16 Cardiff University 83.3 71.0
28 4 University of Oxford 93.2 70.3
29 57 Bishop Grosseteste University 58.5 70.2
30 72 Leeds Trinity University 50.2 68.3
31 59 University of Bradford 55.2 68.0
32 49 Bangor University 61.1 67.5
33 51 Northumbria University 60.4 67.2
34 55 York St John University 58.8 67.0
35 31 University of Reading 72.9 66.8
36 =22 Queen’s University Belfast 77.5 65.9
37 =2 Imperial College London 93.3 65.6
38 17 University of Sheffield 82.5 65.3
39 =37 Heriot-Watt University 68.9 65.2
40 75 University of Northampton 47.6 64.9
41 64 St Mary’s University, Twickenham 52.7 64.4
42 28 University of Dundee 75.7 64.3
43 40 Royal Holloway, University of London 67.8 64.1
44 26 University of York 76.5 63.9
45 =60 Bath Spa University 55.1 63.7
46 44 City University London 65.7 63.0
=47 71 Brunel University London 50.6 62.2
=47 63 University of Worcester 52.9 62.2
49 27 University of Southampton 76.3 61.7
50 25 University of Nottingham 76.7 61.2
51 65 University of Hertfordshire 51.6 60.9
52 18 University of Warwick 79.5 60.6
53 =76 University of Derby 47.1 60.5
54 81 Manchester Metropolitan University 45.8 60.0
55 =69 Sheffield Hallam University 50.7 59.2
56 21 University of Glasgow 77.7 57.7
57 =89 Birmingham City University 40.7 56.8
58 48 Robert Gordon University 62.4 56.7
59 66 University of Hull 51.5 56.6
60 78 Aberystwyth University 46.7 54.7
61 67 University of the West of England 51.2 54.3
62 =76 Edge Hill University 47.1 54.1
63 30 University of Manchester 73.4 53.8
64 80 Liverpool John Moores 46.3 53.0
65 29 University College London 74.4 52.5
66 =83 University of Sunderland 43.0 52.4
67 35 Queen Mary University of London 70.1 52.2
68 94 University of Greenwich 36.0 49.8
69 =83 Anglia Ruskin University 43.0 49.6
70 82 University of Brighton 44.5 49.1
71 112 Southampton Solent University 24.5 48.6
72 104 Middlesex University 30.7 48.5
73 99 Kingston University 33.3 48.4
74 =22 University of St Andrews 77.5 47.7
75 93 University of Gloucestershire 37.8 47.4
76 73 Plymouth University 49.0 47.2
=77 74 University of Chester 48.5 46.7
=77 46 University of Leicester 63.2 46.7
=77 =89 Teesside University 40.7 46.7
80 41 University of Liverpool 66.8 46.6
81 39 London School of Economics and Political Science 68.1 46.0
82 96 Goldsmiths, University of London 34.8 45.6
83 36 King’s College London 69.5 45.3
84 58 University of Stirling 56.1 45.1
85 68 Ulster University 50.9 44.3
86 108 Staffordshire University 27.8 44.1
87 32 University of Bristol 72.4 43.8
88 87 Canterbury Christ Church University 41.8 43.5
89 =42 University of Edinburgh 66.7 42.3
90 88 Liverpool Hope University 41.6 41.1
91 =60 Soas, University of London 55.1 40.9
92 91 Newman University 40.4 40.7
93 52 University of Aberdeen 60.2 39.9
94 95 Bournemouth University 35.6 39.8
95 47 University of Strathclyde 62.8 39.6
=96 79 Edinburgh Napier University 46.4 39.5
=96 101 University of Roehampton 31.9 39.5
98 103 University of Cumbria 31.0 38.9
99 110 University of South Wales 25.7 38.0
=100 =69 Glasgow Caledonian University 50.7 37.6
=100 100 University of Salford 32.5 37.6
102 107 Bucks New University 28.0 36.9
103 92 University of Central Lancashire 37.9 36.6
104 98 Cardiff Metropolitan University 34.0 36.4
105 109 University of Bedfordshire 26.9 35.9
106 86 Abertay University 41.9 35.5
107 102 University Campus Suffolk 31.2 34.3
108 85 University of the West of Scotland 42.5 33.1
109 105 London South Bank University 28.9 32.3
110 113 University of Westminster 23.9 31.9
111 111 University of St Mark and St John 24.7 31.4
112 115 University of Bolton 22.0 30.0
113 106 Leeds Beckett University 28.2 29.7
114 114 Glyndwr University 22.1 27.9
115 117 University of West London 17.3 24.6
116 116 University of Wolverhampton 18.0 23.5
117 118 University of Wales Trinity Saint David 14.7 16.7
118 120 University of East London 0.0 6.7
119 97 Queen Margaret University 34.7 6.6
120 119 London Metropolitan University 0.5 0.0

Notes: Graduate destination data are drawn from the Destinations of Leavers from Higher Education Survey 2013-14. Completion data are drawn from Higher Education Statistics Agency performance indicators for 2013-14. Student satisfaction scores are drawn from question 22 of the 2015 National Student Survey, which asks about overall level of satisfaction with course quality. The overall benchmarked TEF score is based on the distance from benchmark in each of the three categories, weighted equally.

Read more about
Read more about: 
Teaching excellence framework (TEF)
Employability
Politics
Rankings
Students
Teaching and learning

POSTSCRIPT:

Print headline: A new perspective

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Register
Please Login or Register to read this article.

Related articles

Related universities

Reader's comments (8)

#1 Submitted by HE professional on June 23, 2016 - 9:25am
There is so much wrong with the way this is calculated I don't even know where to being. Sensationalist rubbish...
#2 Submitted by a.kolb on June 23, 2016 - 3:27pm
That may well be; I find it difficult to tell what was included and how it was weighted. But the key question is, how similar will be the HEFCE metrics be to the ones that are used here? I.e. are these results relevant to the real TEF?
#3 Submitted by ABC on June 23, 2016 - 2:10pm
I completely agree. Slow news day at THE? And however the actual TEF is calculated, will it really "send shockwaves not just through the UK but across the world”? Come on. The bottom line is league tables and information that potential studnets use to decide which institution to go to is what counts. This information already exists. Whatever the TEF leagues look like, and they may well reflect some of the patterns imagine above, impact rightly or wrongly will be much less. Also poor, or what is perceived as poor teaching exists across all (types of) instituitons.
#4 Submitted by a.m.hayes_285834 on June 23, 2016 - 4:32pm
I agree with all comments above, but also, why are the results considered in terms of the subject mix and entry qualifications when we want to assess the impact of the type of student that is recruited? Don't international students matter here? I have recently written about it in my article that points to a fundamental problem with the TEF as a national tool that legitimises subordination of international students in recent moves and changes to higher education. International students are an important group that have an impact on the TEF results, but they are not considered in any of the discussions regarding the TEF.
#5 Submitted by deevybee on June 23, 2016 - 7:25pm
I would be grateful for further explanation of how these scores were arrived at, because there appear to be some anomalies. The pdf table helpfully gives the breakdown of scores for the 3 components, i.e. destination, student satisfaction, and completion rates. We are told that: “Giving each of the three metrics equal weighting, an absolute score for each institution is generated”. This implied that before any benchmarking was done, the column called Overall Absolute Score should be derived from the sum of the three components. I can see it has been scaled to range from 0 to 100, but that should not alter the relative standing of institutions. However, we have several instances where institutions with the same total score have different values on Overall Absolute Score. For example, if you average the non-benchmarked scores for Graduate Destinations, Completion and Student Satisfaction for Surrey, Sussex, Leeds, Cardiff and UCL they all are the same, with mean score of 87.33. Yet their ranks on the non-benchmarked scale are given as 12, 20, 14, 16 and 29 respectively. Either there is an error in the tables, or the description of the computations is incomplete. I wondered whether there had been some adjustment for the size of the institution. As I noted in a Times Higher blogpost, this kind of adjustment penalises large institutions because their raw scores have a smaller standard error (see: https://www.timeshighereducation.com/blog/nss-and-teaching-excellence-wrong-measure-wrongly-analysed). I plan also to look at the impact of benchmarking on your results, but suffice it to say, if your tables have been computed using the same kind of benchmarking as I described in my blogpost, they are highly questionable. When evaluating institutions, one can make a case for benchmarking against student intake characteristics when the measure is something like completion rates - although even there, one wonders whether the average student (for whose benefit we are told this whole exercise exists) is more concerned about absolute or benchmarked estimates of outcomes when considering where to apply. But for student satisfaction, benchmarking seems to imply that we must expect that students with lower entry level qualifications will be less satisfied with their courses, and adjust for this. This seems to go against one of the stated goals of the White Paper, which is to encourage institutions to provide courses that are supportive of those from non-traditional backgrounds.
#6 Submitted by nicki.horseman_237466 on June 24, 2016 - 2:27pm
Thanks for your interest. The method applied to generate the score is to take each of the individual indicators (absolute or relative to benchmark), z-scoring them to bring them to a common scale and then combining with equal weights to produce either an absolute or relative score. The data contains more decimal places than presented in the article and there is no adjustment for size of institutions. The benchmarks are established benchmarks derived by HEFCE/HESA. In terms of the use of entry qualifications as a factor in the benchmarks, it is used in Completion and Graduate Destination benchmarks but not in Student Satisfaction. This is as proposed in tthe TEF Year Two technical consultation https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/523340/bis-16-262-teaching-excellence-framework-techcon.pdf
#7 Submitted by deevybee on June 23, 2016 - 8:15pm
I've just looked at the data behind that figure purporting to show "only a weak correlation between REF and TEF performance." Essentially, you can show pretty much anything, depending on how exactly you compute and select your scales. For the TEF and REF indices used in the figure, the correlation between the two is .359. If you take the absolute TEF, without benchmarking, the correlation with the REF is .732. So can we really conclude, as the government would have us believe, that there is a reciprocal relationship between research excellence and teaching excellence? In your piece it is suggested that: “Some of the institutions which are doing best on the measures we put together are smaller, campus-based universities,” she said. “It may be that they have a stronger connection with their students and more evidence of a community.” Alternatively, it could be that they benefit greatly from use of a benchmarked index, especially if the TEF index is biased in favour of small institutions because of the larger standard error around the estimates of their scores. If the TEF is going to gain the confidence of institutions, it's going to have to be based on better measures than this.
#8 Submitted by Classic on July 7, 2016 - 7:47am
Only an OFSTED inspection will check the accuracy of information collected by universities. Keele University for example only had 4% of its postgraduate's filling in the survey conducted by an independent company. We know this because the independent company sent out an EMail a week before the survey deadline encouraging more students to complete the survey. The next thing we saw were large banners everywhere stating 71% of students thought Keele was outstanding. Were the 4% employed by Keele in some capacity or other I wonder? Did the other 96% fill in the survey form within a week? What do you think. Discuss.

You might also like

Sponsored

Featured jobs

See all jobs