Could online exams transform how we think about learning?

Forced into a snap transition into online teaching by the coronavirus pandemic, academics may dismiss remote assessment as another necessary evil. But one pioneering online institution predicts that many will be pleasantly surprised by the results of the shift, while another says the change has brought a more progressive view of assessment methodology.

The experiences of these institutions address some key specific questions, such as: how do you stop people cheating in online exams? And their accelerated journeys into online assessment raise an even bigger question: what is learning about and how should we measure it?

Australia’s University of New England (UNE) has expedited the last year of what was supposed to be a four-year changeover to distance examinations, with all assessment now conducted remotely. While the capability has helped it manage the coronavirus, UNE says this has not been the main benefit.

“The benefit to us is in student retention,” said Jennifer Lawrence, UNE’s programme director for academic success.

“We’ve measured the impact of online exams on retention at UNE at somewhere around 2 per cent, or several hundred students. The flexibility has a noticeable impact on their success long term.”

Ms Lawrence said that when sickness or accidents kept students away from paper-based exams, the frustration of waiting for the next opportunity – typically during the following trimester’s exam period three months later – prompted some to leave. “We were losing many of those students,” she added. “Now we can say, ‘how about next week?’”

The flexibility also allows opportunities for students to choose their own pace in their courses and assessment. “We come to the point of exams on demand,” Ms Lawrence said. “If students want to go really fast, they can. When they’re ready to sit, they say, ‘I’d like my exam now.’”

The University of Western Australia (UWA), a relative novice in distance education, has “done a complete U-turn” in rapidly migrating its undergraduate courses online. “We’re also changing a lot of our assessments [to] take-home exams,” said Vaille Dawson, deputy head of school education in the Graduate School of Education.

“The purposes of assessment are so that students can show they’ve achieved the outcomes, and so that you can rank them. Showing you’ve achieved the outcomes is probably more important. Changing to take-home-type assessments is valid and reliable.”

Reggie Kwan, provost of the Open University of Hong Kong (OUHK), said 30 years’ experience with distance learning had informed the institution’s shift to online assessment. OUHK conducted almost all of its assessments online during the exam period in late 2019 and has refined the process for final examinations in May.

Professor Kwan said only about half of OUHK’s online assessments mimicked traditional sit-down exams. The rest are conducted via papers, take-home tests or other assignments.

This marks an important shift in a society preoccupied with testing. “Usually in Hong Kong, final exams are a big deal and may make up 50 to 60 per cent of a total mark. Whereas it may only be 20 to 30 per cent in the US – and that’s a good thing,” said Professor Kwan, who has worked in the Montana University System.

“We have to look holistically at assessments,” he continued. “Will students learn what they need to learn? And can we gauge their abilities? That’s what’s important. The situation has forced academics to think: what is learning about? Is it about a paper-and-pencil exam?”

Unexpectedly, the shift online has increased class participation. Attendance is 20 per cent higher than in face-to-face classes, OUHK has calculated.

Ms Lawrence said that when UNE introduced online exams, it had looked for wildly unexpected results that could have indicated cheating. “We didn’t get sudden dramatic increases,” she said. “Students who were on track to fail still failed; students on track to succeed still succeeded.

“We have also looked at options for giving students feedback on exams – exams for learning, not just exams of learning,” she added. “In a paper-based setting [that] is difficult to manage logistically.”

Professor Kwan said Hong Kong students had not traditionally received copies of their marked exams. That may change with online assessment.

However, assessments for disciplines such as nursing cannot realistically be done online. “Nurses need a certain number of hours of practicum to be accredited. Nobody wants to be treated by a medical professional who has not practised doing an injection,” Professor Kwan said. So OUHK has combined online testing with voluntary, small-batch, in-person assessments for some subjects.

At UNE, surprisingly few subjects have proved impossible to examine online. Ms Lawrence said the company contracted to provide virtual invigilation services, US-based ProctorU, even conducted online fitness assessments of people studying to become personal trainers. Among the subjects that UNE offers, only Chinese calligraphy has stumped attempts to test it online. 

While some subjects lend themselves to online examination more readily than others, the same applies to students. For those with hardware or internet deficits, physical disabilities or other problems, forms of assessment rather than examination can be the best approach.

UNE has experimented with another option: do-it-yourself exams. Students “write the questions [and] a justification for each of the questions, and then an answer schema”, said Ms Lawrence. “It assesses the same kinds of knowledge. They have to have quite a deep knowledge of the subject area to do that well.”

Ms Lawrence said that approach had proved fruitful, for two reasons: “Firstly, it’s legitimate assessment and it’s equitable.

“Secondly, students who were wavering around the online exam – because they were nervous about the technology – looked at the alternative assessment task and said, ‘Actually I probably magically can do my exam online after all.’ It nudges them in that direction.”

While UNE stresses the pedagogical advantages of online testing, it also saves significant cost and effort. Under the paper-based model, the university routinely conducted exams in some 400 centres in Australia, China, Europe, the US and elsewhere.

“We’ve had exam centres in Antarctica [and] on defence force ships,” Ms Lawrence said. “I’m saving A$60,000 [£30,000] on postage, not to mention the printing bill for the exam papers.”

While such expenses have now been supplanted with a ProctorU bill, staff who previously organised exams around the planet can now focus their attention on the online process and help manage one of universities’ major fears about remote exams – cheating.

OUHK initially considered high-tech solutions such as close monitoring of students via live cams, but “decided not to go down that route”, Professor Kwan said. Instead, educators had been given the liberty to choose their own methods.

However, UNE’s remote exams boast high-tech features including webcam observation and remote access to students’ computers, with facial recognition and biometrics to stop impersonators taking the tests. Integrity expert Cath Ellis, associate dean of education at UNSW Sydney, said other technological safeguards were available.

They included two-factor authentication and eyeball tracking. “If you’re constantly looking at the ceiling or to the side of the screen, that’s telling [examiners] that there might be notes plastered there,” Dr Ellis said.

“My hunch is that the assurance of academic integrity via virtual proctoring, particularly with the very expensive models, would be better than you would get in an exam hall.”

For its exams-on-demand model, UNE uses question banks to guard against students informing each other about exam content. “Not every student gets the same questions,” Ms Lawrence said. “They’re assessed against the same skills, but there are subtle and sometimes dramatic differences.”

Similar problems existed when UNE conducted tests in different time zones, she added. “It’s an illusion that every student is sitting their exam at exactly the same time,” she continued. “We didn’t have strategies to manage that at the time [before the pandemic]. Many institutions don’t.”

She said the incidence of “low-level cheating”, such as students having textbooks on hand, was likely to rise in online exam settings. But such problems were easily nipped in the bud, she argued: “The invigilator can just ask the student to put it away, rather than have an academic misconduct case later on.”

Frank discussions on the expectations of students during exams, and the materials they could bring with them, helped separate their “exam anxiety” from their “technology anxiety”.

Students “have a chance to practise with the technology first”, Ms Lawrence said. “It alleviates a lot of the stress of the exam situation. Anecdotally, we’ve found that has a positive impact on the emotional reactions that lead to cheating in the first place.”

She said universities grappling with the sudden need for online assessment should view it as “a communication and change management project rather than a technology integration project”.

Many universities were focused on the “tool”, be it ProctorU or some other service provider. “They run into trouble when they forget that they don’t just have to buy the car – they have to train people to drive it,” Ms Lawrence said.

john.ross@timeshighereducation.com