The law of averages produces poor results

Our honours classification system is an anachronism. It is an inadequate judge of ability and must go, says Mantz Yorke

Why do 17 per cent of students of engineering and physical sciences obtain first-class honours whereas only 5 per cent of law students do? The disparity probably has more to do with assessment methodology than with students' ability levels.

Work conducted by the Student Assessment and Classification Working Group over more than a decade has pointed to persistent differences between subject areas in both the grades awarded at module level and the award of honours and has prompted wider research into the factors that influence assessment outcomes.

This research suggests that the honours degree classification is less reliable than many believe, despite the manifest commitment of academics to fairness in the assessment of their students. It has outlived its usefulness and should go.

The weakness in the classification system reflects variability in assessment at all levels, from that of the individual assignment to the regulations adopted by institutions.

Assessments necessarily sample from the curriculum, and the sampling will to some extent influence the grade awarded. Assessments in higher education are "local" to the institution (even part of the institution) concerned, in contrast to national examinations such as A level.

Assessment criteria, which contain a mix of norm and criterion-referencing, are generally fuzzy; this leads to the exercise of judgment by the assessor except in cases - notably in science-based subjects - where the work can be graded unambiguously against a template. Assessors of essay-type work may grade in broad bands and use finer distinctions to indicate where in a particular band the standard of the essay lies. Research suggests that academics' approaches to grading vary widely.

With a growing interest in employability and the exercise of "soft skills", grading is problematic because of the context dependence of achievements, such as the ability to work collaboratively. It is not normally feasible to assess a student's performance across a range of situations and hence to grade prowess reliably.

Rarely reported are the conditions under which the student approached an assigned task. For some, a task may be structured in detail, whereas others may have had to cope with a similar task without such support. A grade tells nothing about this.

An honours degree classification is determined by combining results from a range of different assessment tasks that have been evaluated in various ways with varying levels of precision.

This often involves the averaging of percentages but without taking account of the distributions from which the percentages came (did everyone studying the module score high, or was an individual's performance outstanding in relation to the performances of peers?).

The averaging is of particular significance at the important boundary between upper and lower second-class honours, where relatively small changes to marks can tip the balance either way. For example, one cannot say with confidence that a student with an average of 60 per cent is a degree class "better" than one who achieved, say, 58.5 per cent.

A student's classification is influenced by their choices of subject and module. The recent report Outcomes from Institutional Audit: Assessment of Students from the Quality Assurance Agency indicates that the classification of joint and combined honours degrees presents particular problems because of inter-subject variation in the approach to assessment.

Classification depends on rules established, quite properly, by autonomous institutions to suit their particular purposes. However, classification methodology is not consistent across the higher education system because institutions differ in the rules and regulations that they use to determine honours classifications.

Simulations using real data have shown that up to one in six classifications could be different if another institution's rules were applied.

When the higher education system in the UK was smaller and more homogeneous in a number of respects, the honours degree classification was arguably less of an issue than it is today.

The variety of degree programmes now on offer, with their different kinds of student demand, points to the need to do more than reduce the multidimensionality of a graduate's performance to a single index of achievement that is employed across the whole of the higher education system.

The honours degree classification loses too much information. It is too crude to merit preservation.

Mantz Yorke is visiting professor at Lancaster University.