Must we sell our souls to make a bigger impact?

Source: Miles Cole

The pamphlets imply that social science is able not only to answer factual questions but also to provide practical evaluations of what is wrong

There have been some high-profile examples of the mis-selling of goods and services, such as insurance, that have no value to the people buying them. More pervasive, of course, is over-selling: the making of exaggerated claims for products and services. Recently, I have begun to reflect on whether some recent efforts to promote social science to politicians and the public could fall into this category.

Considerable effort and resources have been devoted in the past few years to advertising the benefits of social science. The motivation, in large part, has been to reduce the threat to funding posed by austerity programmes. But there are dangers and difficult dilemmas involved.

As illustration, take the “Making the Case” pamphlet on crime produced by the Academy of Social Sciences and the British Psychological Society as part of the Campaign for Social Science. One problem is the way that research findings are presented as conclusive, with little attention to methodological doubts to be found within the research community about their validity. For example, it says that the Cambridge Study in Delinquent Development “provided evidence about risk factors for offending, especially those measured in childhood such as impulsiveness, low school achievement, poor parental supervision and disrupted families”. But measuring different factors suspected of increasing the chances of delinquency, and assessing their effects, has long been recognised as a difficult and controversial matter.

Some researchers insist that causal relationships can be discovered only through experimental investigation, while others emphasise the complexity and contingency of the processes involved and the near impossibility of prediction. This sort of research has also been challenged on ethical or political grounds as encouraging a tendency to blame impoverished families and communities rather than addressing the wider social causes of crime and delinquency. Similar doubts could be raised about the rest of the research in the pamphlet, as well as in the others in the “Making the Case” series.

It is also significant that the pamphlets present the findings of particular studies or programmes of research whose validity is necessarily likely to be more uncertain than a review of all research relating to the topics concerned. This problem also arises with current pressures on researchers to maximise the impact of their studies. Common advice, such as that offered by Cardiff University’s Jonathan Shepherd (“Say what? If you want your research to influence policy, speak plainly”, 8 January), is to produce short, accessible summaries and to drop caveats; yet some of these caveats will relate to methodological cautions. In recent times there has been a great deal of emphasis, instead, on systematic reviewing of research literatures to establish a reliable knowledge base for policymaking. This is the rationale behind the government’s network of What Works centres, for instance.

A second kind of mis-selling is also worth mentioning. The pamphlets imply that social science is able not only to answer factual questions but also to provide evaluations of what is wrong, and recommendations about what should be done. One example is the claim that a study presenting new evidence of how and why people stop offending “showed that successful supervision of offenders needs to be based on working in and through the relationships that lie at the heart of the process of giving up crime”. This may well be true, but such recommendations necessarily rely on value assumptions about what is desirable – regarding both the seriousness of particular offences and the appropriateness of particular strategies for encouraging desistance. These assumptions can be contentious, and cannot be validated by research alone.

Even judgements about whether discouraging people from offending is desirable can vary. In the case of many kinds of offence, there would be little disagreement. But there are some examples where there may well be: occupying public areas to promote a cause, stealing food to feed one’s family or squatting in empty houses would be cases in point.

Even more serious problems arise when it comes to evaluating and recommending policies: what should be done is never a matter solely of what would be effective; ethical and political considerations are also involved. Research cannot adjudicate on these matters.

Of course, acknowledging that the findings of the social sciences are restricted to matters of fact, as with recognising the uncertainty that often surrounds their validity, may handicap attempts to make a public case for these disciplines. This is especially true in a political climate where excessive and dogmatic claims are routinely made about what is wrong and what should be done. But perhaps this also signals a vital role for social science. Raising doubts and questions about what are assumed or asserted to be conclusive facts and watertight recommendations could encourage a more careful and deliberative approach to policymaking, from which everyone would benefit.