Wednesday 20 June 2012

'Test, Learn, Adapt' - thoughts on the use of RCTs in social policy

Today, the Cabinet Office's 'nudge unit' has produced a new document advocating the greater use of randomised control trials (RCTs) in the policy-making process.  With 'Bad Science' writer Ben Goldacre on the team, it's received a fair bit of press, so I thought I'd review it and offer my own opinion on their key argument.

The argument
The thesis behind the paper is simple.  RCTs have been used in a range of other fields for some time - medicine, international development and business for example.  However, they have not been adapted to the policy-making process so enthusiastically.  Despite the early New Labour focus on 'what counts is what works', governments maintain the habit of ignoring scientific evidence on policy interventions.  The authors argue that this should change: if it does, we might get better policies and at a significantly cheaper cost.

Sounds too good to be true?
That's because it is.  In my view, the report rehashes old arguments about the use of 'scientific' evidence  in social policy.  The authors appear to come from a very one-sided scientific perspective: unaware of the long-established difficulties - particularly in social policy - of the interaction between science and society, politics and morality.  In fact, in their discussion of the 'myths' about RCTs, the authors focus exclusively on practical or economic objections, rather than any philosophical doubt about using scientific evidence to decide on policies.

The authors offer a deeply tempting and seductive view: that we can determine what policies we choose by scientific methods, removing age-old ideological debates about the Good Society once and for all.  Not only is it wrong to equate the objectives of social policy to other institutions, like medicine, but to my mind it seems quite undesirable.  Sure, we can use scientific methods like RCTs to choose between different policy interventions.  But first we have to decide what type of interventions we want in the first place.  And that requires debates and deliberations that science simply cannot answer.


Sunday 17 June 2012

Is the UK alone in turning against the welfare state?

In a paper I recently had published in the journal Social Policy and Society,  I looked at longitudinal data from the British Social Attitudes (BSA) to explore how attitudes to the welfare state - and in particular the unemployed - had changed over the past few decades.  The data show a strong and profound shift in attitudes against welfare.  Whilst once a majority of us tended to support more spending on benefits, there is now a solid consensus against extra spending on social security (especially for 'undeserving' groups like the unemployed).

In my paper, I try to offer an explanation for why this happened.  In short, I argue that the growth in anti-welfare attitudes has followed the deep and significant reforms to the welfare system that have taken place since the mid-1990s: stronger conditionality, sanctioning and the growing emphasis on personal responsibility.

However, this is not a particularly easy argument to prove one way or another.  I argue that the main basis supporting this theory is the timing of the attitudinal shift: around 1997/8 (or when New Labour came to power with a promise to change the welfare state).

Is this the only explanation?

An alternative hypothesis is that the UK has not been alone in its changing attitudes to welfare.  For example, it could be argued that anti-welfarism here is part of a broader, global trend in attitudes that come as a function of the shift to post-industrial (and largely neo-liberal) societies in the rich world.

One way to test this argument is to look at comparable, cross-national data on attitudes towards the welfare state.  This is not particularly easy: it's difficult to find attitudinal data that has been collected consistently over time and for a decent sample of countries.  There is one dataset though - the International Social Survey Programe - that might be able to give us some clues.

One question the ISSP asks concerns attitudes towards state generosity for the unemployed.  The question is:

"The Government should provide a decent standard of living for the unemployed" Do you strongly agree/agree/neither/disagree/strongly disagree?


This question was asked to respondents in five countries in both 1987 and 2010: Austria, Australia, Germany, the UK and the US.  This is a small sample but an interesting one: it contains three countries from 'liberal' welfare regime types (Australia, the UK and the US) and two from 'conservative' ones (Austria and Germany).  Comparing these five countries for the two time-points should allow us to see if there are any similarities (or differences) between the regime types in terms of changing attitudes towards supporting the unemployed.  It should also suggest whether shifting attitudes are universal, rather than a sole feature of British society.

A universal shift?


The results are quite surprising.  Here is a table that shows the proportion of respondents stating that they either 'strongly agree' or 'agree' that the Government should provide a 'decent standard of living for the unemployed':









A quick glance should tell you that only the UK and Germany have seen a drop in support for supporting the unemployed.  Indeed, even the US has seen a (slight) increase in agreement with decent public support the unemployed.  Here's a graph showing the percentage change between 1987 and 2010:















This kind of data is by no means conclusive: it's based on just one question for a sample of five countries.  But it does suggest two things.  First, that the shift against welfare in the UK - evidenced in domestic surveys like the BSA - is also evident in global, comparative surveys.  Second - and perhaps most importantly - other countries have seemingly had completely different experiences.

This is important.  In Britain, it's often implied by politicians and reformists that welfare retrenchment is inevitable: that it comes part and parcel of a new kind of economy and a changing society.  According to this theory however, other countries should also be undergoing the same kinds of cultural shifts.  The data shown above suggest the picture is more complex.  And that the future of the welfare state is much more uncertain than many of us think.