Examiners give hugely different marks

Source: Rex

Grading gamble: the paper says marking is a ‘judgement, not a measurement’

Undergraduates awarded low scores for their work may receive top marks if they are graded by another academic, a study suggests.

As part of the study into inconsistent marking, 24 academics in four disciplines were recruited from 20 UK universities and each asked to mark five pieces of student work originally graded as borderline 2:1 or 2:2 by universities.

But many of the academics – who had all worked as external examiners – ended up awarding hugely different marks, according to the study published in the journal Assessment and Evaluation in Higher Education.

Some essays were given a first, while others received a third-class mark, says the study, which is titled “Let’s stop the pretence of consistent marking: exploring the multiple limitations of assessment criteria”.

Indeed, very few of the assessors in history, chemistry, nursing and psychology believed that the students’ assignments were borderline, with only seven placing all the work within two adjacent grade bands.

Nine of the 20 assignments were also ranked both best and worst by different examiners, it says. Only one of the pieces was ranked the same by all six assessors, it adds.

The publication comes midway through a fundamental review of quality assurance by the Higher Education Funding Council for England, in which the role of external examining has again come under scrutiny.

Madeleine Atkins, its chief executive, has said that she supports external examining but wants to see the process strengthened.

A review in 2011 by Dame Janet Finch, former vice-chancellor of Keele University, found that the process was working well, although concerns have been raised at how different academics apply the same criteria to ensure that academic standards remain broadly comparable across the sector. The paper argues that “a large element of unreliability” would always remain in assessment, despite efforts to moderate work, as grading is a “judgement, not measurement”.

“We should recognise the impossibility of a ‘right’ mark in the case of complex assignments, and avoid over-extensive, detailed, internal or external moderation,” it adds.

Sue Bloxham, emeritus professor in academic practice at the University of Cumbria, who co-authored the paper, said that, except for new markers, we should stop using moderation to achieve the “correct” mark for each assignment because it probably does not exist.

“We spend considerable time moderating these individual assignments, whereas fair and reliable awards are obtained because students are judged on a profile of work assessed by a range of academics,” Professor Bloxham told Times Higher Education. “Moderation should be retained only for heavily weighted items such as final-year dissertations, and that should be really thorough because of the impact on students’ awards,” she added.

The report concludes: “Universities need to be more honest with themselves, and with students, and help them to understand that application of assessment criteria is a complex judgement and there is rarely an incontestable interpretation of their meaning.”

jack.grove@tesglobal.com