Teaching evaluations ‘mirror students’ gender and culture’

Students rate first-time lecturers who share the same gender and heritage as them above seasoned academics from different backgrounds, according to a study.

Australian researchers analysed a massive dataset to explore whether student evaluations of teachers were driven more by bias than teaching effectiveness. The findings suggest that student evaluations are so unreliable a measure of teaching quality that “bias correction to recalibrate the scores” is warranted, the authors say.

The study, thought to be the first to investigate the influence of racial as well as sexual prejudice on student evaluations, has been published in the journal Plos One.

The team analysed more than half a million student evaluations of about 2,400 courses and 3,100 teachers from five University of New South Wales faculties over a seven-year period. It found that evidence of gender bias was most pronounced in science – where men dominated both teacher and student numbers – and smallest in arts and social sciences, where women predominated both in faculty and enrolments.

But student surveys in the arts, where Australians massively outnumbered international students, proved likely to rate locally born teachers above foreigners.

The interplay of gender and race proved particularly harmful to the ratings of female academics with non-English speaking backgrounds. Male native English speakers were more than twice as likely to attract higher scores in science, 82 per cent more likely in business and 61 per cent more likely in engineering and medicine.

They also proved most likely to attract the highest possible grading in every faculty except engineering. The researchers said that male engineering academics tended to draw lower ratings than men in other disciplines.

In the business faculty, domestic students – who were mostly male – appeared to give higher ratings to male academics, while the female-dominated international students favoured female teachers.

“The results support the argument that we unconsciously preference people who are more similar to ourselves, regardless of whether that similarity arises through gender or culture,” the paper says.

The researchers found that signs of bias vanished when they analysed evaluations of courses rather than teachers. The odds of favourable assessments were stacked against female science academics in teaching evaluations, but 50:50 in course evaluations. “Biases creep in when students evaluate the person, not the course,” the researchers conclude.

They acknowledge that, as an observational study, the research could not prove that prejudice was responsible for the differences. But there was no other credible explanation.

“The only plausible causes are either that females are generally bad teachers across a large population, or there’s bias. The same argument can be made for teachers who have non-English speaking background,” the paper says.

UNSW statistician Yanan Fan, who led the study, said the findings had major social implications. “Over 40 per cent of the Australian population now goes to university,” she said. “Graduates may carry these biases with them into the workforce.”

Co-author Emma Johnston, UNSW’s science dean, said more recruitment of female professors, committee members and community leaders could “help shrink these biases”.

“We need to continue to support women at all levels of academia in STEM across Australia, in order to smash stereotypes that create the partiality that exists within our community,” she said.

john.ross@timeshighereducation.com