Tide turns on ‘inherently biased’ student evaluations of teaching

More universities are being urged to scrap or revise end-of-module student evaluations as concerns mount about the toll they take on lecturers’ mental health.

Institutions in the UK, the US and Australia have moved away from using the once-pervasive exercises in recent years, but they remain commonplace despite a growing body of evidence that they often contain abusive comments and are statistically flawed.

An alternative is to run a more informal midpoint “check-in” that focuses less on quantitative data and more on what could be changed in the remainder of the course, said Athanasia Daskalopoulou, a senior lecturer in marketing at the University of Liverpool.

She authored a paper published recently in Studies in Higher Education in which UK academics recount being called “fat” or facing disparaging comments about their dress, age and accents.

The University of Southampton scrapped compulsory end-of-module questionnaires last academic year, and special permission must now be obtained to use them. If granted, course leaders are instructed to “take care” over how this is done “in recognition of the concerns regarding conscious and unconscious bias”.

“We have moved instead to mid-module questionnaires in most circumstances,” a spokesman said. “The reason for this is so that any suggestions students make may help to improve their own experience, in addition to benefiting those in the next cohort.”

Newcastle University has also moved to a similar system, with informal check-ins focused on aspects of a course where things can be adjusted immediately to suit students’ needs.

Mid-point evaluations are common across the sector, but most universities use them in tandem with a more formal university-run exercise at the end of a course.

It is these that have proved most controversial, said Dr Daskalopoulou, because the data is often shared widely, and can be used in appraisals and decisions about promotion.

“In mid-point evaluations, students direct their energies towards what they see as working and not working so they can benefit in the learning experience; whereas at the end it is usually an evaluation of how they felt about the person teaching, or if they were happy with their grade,” she said.

Campus resource: Should we be aiming for student happiness or student satisfaction?

Dr Daskalopoulou’s study, which features interviews with academics at UK business schools, sought to discover how people were affected by the surveys on a personal level.

One participant, a white, female professor given the pseudonym Sara, says: “I got fat. I got that put on two evaluations…Then just negative comments about what I look like...the ones that really stick in my mind are the ones about my weight…it kind of really hurts.”

Another, known as Mark, says: “It takes one comment to kind of question yourself and your own career.”

Dr Daskalopoulou predicted that, as evidence emerges from the institutions that have moved away from end-of-module evaluations, “maybe more universities will look to find alternative ways of hearing the student voice without linking evaluations to other aspects”.

Alongside switching to less formal evaluation processes, Dr Daskalopoulou said some universities were also considering removing the anonymity of students who make very offensive comments so that they are aware that there might be repercussions, although she said this brought clear ethical concerns and was not always straightforward.

Troy Heffernan, a senior lecturer in education at the University of Manchester and co-author of a study published last year that argued that universities’ use of student evaluations left them vulnerable to legal action from distressed staff, agreed that some institutions were moving away from their use.

He said that, via his consultancy work with more than 30 universities worldwide that have reappraised their evaluation processes in the past two to three years, most have moved to adopt a system of peer reviews of teaching and internal revisions of course content.

“These methods are time-consuming, but for universities who have admitted that student evaluations are inherently biased and bad for everybody – though worse for anyone who isn’t a white, straight, able-bodied, heterosexual, middle-class man – this is how you check course content and teacher quality without relying solely on student populations,” he said.

But Dr Heffernan cautioned that while some universities were evolving, they still represented only a fraction of the total number of institutions worldwide, and it should be seen as a long-term process.

“While we are seeing a change, at this point the change is only small when this was a practice that was taking place in a huge majority of the globe’s 16,000 higher education institutions,” he said.

tom.williams@timeshighereducation.com