Universities are central to the world of policy

Towards the end of his career, in 2005, the US political scientist Harold Wilensky wrote a journal article in which he tried to explain – to himself as much as to others – how and when social science had an impact on policymaking.

He bemoaned what he saw as a tendency in the US for the government to invest in short-run empirical studies and policy evaluations in preference to basic research and integrated, multidisciplinary thinking. More generally, he observed a tighter fit between research and policymaking in the corporatist countries of northern Europe than in the liberal market economies. But he ended on an optimistic note, arguing that, in the long run, it is fundamental social science rather than short-term evaluations that influence “elite and mass perceptions of social reality”: in other words, how a society thinks about itself.

As the UK journeyed from Keynesianism to Thatcherism in the second half of the 20th century, it traversed Wilensky’s political economic categories of corporatist and liberal market policymaking. In the post-war decades, policymakers looked to professionals, social partnership institutions or democratically elected local authorities to shape and implement policy. Scientific and bureaucratic expertise was highly valued. Ministers frequently established Royal Commissions or committees of the eminent and expert, like that on higher education chaired by Lionel Robbins in the early 1960s, to undertake long-range policy development.

But by the turn of the century, Royal Commissions had all but disappeared, in favour of task forces and policy tsars. The 1980s had brought in “new public management”, curtailing the distrusted autonomy of the professions and sweeping away social partnership and the search for consensus. Power was increasingly centralised in Whitehall, and public services were opened up to inspection, quantification and quasi-markets. This reached its apogee in the “delivery state” of the New Labour era – when investment in and reform of public services was the government’s central objective.

Today, the landscape of research and policy development is again transmuting. Targets and indicators have been stripped back, spending has been cut, and power is being devolved to the UK’s constituent nations and cities. The high-water mark of new public management has passed – but there is little sign of a return to post-war corporatism. Instead, technology is opening up policy to greater contestability and scrutiny, and new forms of quantification have emerged with the spread of big and open data. “What Works” centres have been created for large chunks of government activity, such as early years and urban policy. Policy actors have multiplied, from the burgeoning thinktank scene, to new “policy labs” and innovation teams in central and local government.

The front line of public services is also now a site of research and argument, not simply a receptacle for policy instructions. Websites such as Guerrilla Wire, which syndicates independent blogs, give voice to service users, practitioners and those on the sharp end of public bureaucracy, while professionals use dedicated sites and research networks to shape and share best practice. The Cabinet Office has an “open policymaking” function, with a remit to broaden the range of people civil servants engage with, “using the latest analytical techniques, and taking an agile, iterative approach to implementation”. This is a conscious mimicry of the techniques used by digital service designers: practising policy development using open, networked and trial-and-error means, rather than relying on the traditional formula of a closed submission to a minister, generated from within Whitehall (although that remains the bread and butter of the Civil Service).

All this means that different modes of governing now co-exist within policymaking functions: technocratic and evidence-led, ideological and conviction-led, as well as more open and democratic. It is a complex terrain, and one made more uncertain still by the fragmentation of party politics and the ongoing economic aftershocks of the financial crisis. Across Europe, insurgent parties of the anti-austerity Left and the anti-immigrant Right have disrupted established party systems, breaking down the stable patterns of governance. The UK has been insulated from the full force of these changes by its first-past-the-post electoral system, but it too has moved decisively to a multi-party politics. Policy is not made in a predictable political environment.

If ever there was a linear path between research and policy, ideas and action, it has long since disappeared. Yet Wilensky’s optimism remains valid. For although the landscape of policymaking has changed, the research and knowledge produced in universities remain fundamental to how ideas are shaped and circulate in society. Only universities have the critical mass and interdisciplinary strengths to undertake systemic research and to apply their findings across different policy domains. The search for impact is drawing them closer to policy and is taking academics out into wider public and political debate.

That has made universities more open to people like me, who have spent their careers working on public policy in government and in thinktanks. It is why moving to and fro across the academic-policy bridge is becoming as common in the UK as it already is in the US. It is also why new institutes, like the University of Bath Institute for Policy Research, which I am about to lead, are being set up. As the pressures on policymakers multiply, universities’ role in tackling long-term social challenges and fostering a democratic, public sphere has never been more important.

Nick Pearce is director of the Institute for Public Policy Research. From December, he will become director of the Institute for Policy Research at the University of Bath. He is a former head of the No 10 Policy Unit.