Bring meaning to the tables

Rankings have a way to go before they truly aid student choice, say Mantz Yorke and Bernard Longden

League tables attract scepticism from academics, but their primary aim is to inform prospective students - particularly those abroad with limited access to information. As sources of information, UK league tables well merit the scepticism.

The tables reflect the value judgements of their compilers. The various tables published in newspapers include all the UK universities, but some of them exclude other institutions. Tables also vary in content: research performance is omitted from The Guardian' s tables on the grounds that it has limited relevance to undergraduate study, but it is included by The Times and The Sunday Times . Further, the measures are weighted differently.

Where completion, or "dropout", data appear in the tables, the figures do not distinguish between students who drop out and those who take a break from studying. This disadvantages institutions whose students decide to break their studies.

The more opaque the manipulation of the basic data, the stronger should be the health warning. Take "teaching quality": scores based on assessments of teaching quality have combined outcomes from methodologies that have differed by body, by metric and over time. In the table published by The Sunday Times in 2004, only teaching quality taken as indicating "excellence" was counted, which created a severe anomaly in scaling because it reduced everything less than "excellent" to zero: how should a prospective student interpret a zero score?

In the most recent table published by The Guardian , teaching quality assessments have been dropped in favour of a "staff score" based on staff grade, qualifications and teaching-research split. The weightings given to the components of the staff score appear arbitrary, and the relationship between staff score and the quality of teaching is obscure.

The compilers of tables quite happily add together measures of input (entry qualifications), process (teaching quality) and output (awards, employment) to produce overall scores. One would not judge business performance in this way. Further, research, resourcing and recruitment of students are highly correlated and exert a huge influence on an institution's league table ranking and swamp the contribution of other measures, such as employment.

In choosing an institution, students may want to consider features that league tables do not include, such as institutional size, ethos, locality and the availability of specific facilities. League tables are promoted as a contribution to student choice but they may act to restrict choice - because of the limitations of their coverage.

If league tables were interactive, as The Guardian has started to make theirs, prospective students could add features and create a bespoke table.

This could be supplemented by the information becoming available via the Teaching Quality Information website.

Despite their weaknesses, league tables are here to stay, not least because they are commercially successful. Higher education needs to engage with the compilers - or, to paraphrase The Beatles' Hey Jude , to take a bad song and make it better.

Mantz Yorke is professor of higher education at Liverpool John Moores University. Bernard Longden is director of the Institute for Higher Education Research and Development at Liverpool Hope University College.

Their report Significant Figures: Performance Indicators and "League Tables" will be launched by the Standing Conference of Principals this week.