NSS overhaul shows ‘dangerous’ drift towards graduate outcomes

A major overhaul of the UK’s National Student Survey is further evidence of a “dangerous” drift towards using graduate employment outcomes as the main measure of course quality, it has been warned.

As part of a policy paper on cutting red tape, the Westminster government said that it had ordered the Office for Students to carry out a “radical, root and branch” review of the NSS because of “valid concerns” that it drove down standards, could be gamed by institutions and had results that did not “correlate well with other, more robust, measures of quality” such as employment outcomes.

Paul Ashwin, professor of higher education at Lancaster University, said while “at one level” he could understand the criticism of how some universities have tried to manage their NSS performance, he had been “shocked” by some of the assertions made in the document without firm evidence.

He said that there seemed to be a determination to see course quality only in terms of graduate employment and salaries, which often related to other factors such as students’ social background or where they worked.

“It just seems to be a move to make graduate outcomes the main measure of quality in the sector and that is so dangerous when we know that doesn’t relate to quality,” he said.

Camille Kandiko Howson, associate professor of education at Imperial College London, said although in her view criticisms about gaming were “completely justified”, relying on graduate outcome measures was also problematic.

“The criticism about the correlation with other metrics is less substantiated,” she said, adding that the government’s own graduate employment data showed that “there is a danger in relying on outcome measures that are largely based on student intake to judge quality in the sector”.

However, she added that although a radical change to the NSS would have a “huge impact” given that it had become so “enmeshed” in the sector, it did not mean reform “cannot be for the better”, adding that the survey had hindered “innovation in student feedback, quality enhancement and quality assurance”.

Rachel Hewitt, director of policy and advocacy at the Higher Education Policy Institute, said one of her concerns was how student views would be properly accounted for without a wide-ranging survey.

There was also a risk that a scaled-down NSS in terms of sample size and frequency – which the government seems to want – would be unable to provide data at a granular enough level.

“If you want to look…at NSS data at course level [in an] institution…you need to have quite high sample size – if not a census of all students – in order to break it down at those levels,” she said.

simon.baker@timeshighereducation.com