Learning analytics could help avoid the unwanted consequences of TEF

After a prolonged period of growth, UK universities are entering a new and challenging era. The number of 18-year-olds is on the decline, Brexit has created financial uncertainty and the universities minister, Jo Johnson, recently warned that low-performing institutions may have to cut tuition fees.

Now more than ever, universities have to prove their worth.

While it has never been more crucial to accurately evaluate universities, the three teaching excellence framework (TEF) metrics currently proposed – employment/destination, retention and satisfaction – have been criticised for not actually revealing that much about the quality of teaching. Some argue that the inclusion of satisfaction as measured by the National Student Survey will serve as a proxy for student engagement.

But there is evidence to suggest that more engaging courses can, paradoxically, deliver both stronger outcomes and lower satisfaction. Being academically challenged is not necessarily what makes a student most happy, but it does deliver better education.

Big data can help to overcome this challenge. By tracking the digital footprint of students, “learning analytics” gives lecturers and tutors a rich insight into the level of class engagement on a week-by-week basis: how much students use online resources, participate in forum discussions, take books out from the library, and even the amount of time spent on campus. Using learning analytics to measure student engagement in the TEF would help policymakers to understand if universities are providing a motivating learning environment.

As a report by the thinktank Reform, published today, concludes, learning analytics could also be used to avoid major issues with the two other proposed TEF metrics. Fears have been expressed that the retention metric encourages the implementation of more easily passable courses, and judging quality on the basis of employment/destination could discourage the admission of women and students from BME and disadvantaged backgrounds who tend to earn less.

These unintended consequences could be avoided by the implementation of a learning analytics metric. The data would enable universities to understand better what works for different cohorts, and if students from certain backgrounds were less engaged than others, it would not be difficult to build into the metric a higher score for engaging this part of the student population.

Even when it comes to teaching excellence as currently measured, the use of learning analytics could help. Its potential to improve retention rates has rightly been emphasised. By providing tutors with a dashboard of student engagement, personalised interventions can be designed for students at risk of dropping out. A pilot conducted by the Open University that used this approach saw retention rates increase by 2.1 per cent.

This will not only result in a better TEF score, but also add significant revenue to university finances. The outcome of the pilot alone – applied to just 11 modules –  translates into an estimated £1.2 million in tuition fees for the Open University.

In the current climate, helping universities to secure their bottom line may alone be enough to ensure the widespread adoption of learning analytics. But universities are still some way off deploying these techniques at scale: a survey from last year found that no university vice-chancellors thought the UK was a leader in this area.

With the TEF set to tie increases in tuition fees to performance, it is only fair that a university’s score should accurately reflect its quality. The sooner learning analytics is adopted as a part of the TEF evaluation framework, the sooner this will be the case. 

Emilie Sundorph is a researcher at Reform. Smart Campuses: How big data will transform higher education is available on the Reform website.

Write for our blog platform

If you are interested in blogging for us, please email chris.parr@tesglobal.com