Improving the use of evidence and evaluation in policy making
Reported by Ryan Francis, PhD Student at UCL, Affiliate Student at Downing College, Cambridge
The Centre for Science and Policy (CSaP) hosted a seminar for academics and policy professionals, led by Dame Professor Theresa Marteau and Dr Matthew Gill, exploring how to improve the use of evidence and evaluation in policy making. Marteau is a psychologist and behavioural scientist, Director of the Behaviour and Health Research Unit, University of Cambridge. Gill is a Programme Director at the Institute for Government.
Marteau began the seminar by presenting her primary concern - the neglect of evidence in many areas of policy making. She argued that the political dimensions of decision-making can often eclipse the evidence to the detriment of positive societal outcomes. Marteau further argued that evidence-neglect is a major inhibitor and barrier to achieving global health, wellbeing, and climate change ambitions. She said there is a gap between ambition and outcome, for example, the government ambitiously pledged in 2018 to halve all childhood obesity in England by 2030, yet obesity rates are still rising. This is a cogent and clear substantiation of Marteau’s claim.
“It would be good outcome if childhood obesity rates have not doubled by 2030, the rates have been increasing over the last three years.” – Dame Professor Theresa Marteau
She explained that, in a similar vein, government has overpromised and underdelivered on carbon emission reductions for 2035, increasing healthy life expectancy by 2035 and the eradication of smoking by 2030. Marteau emphasised that these policies are failing, because they do not include enough interventions for which there is evidence for effectiveness in shifting public behaviours en masse at the scale needed. Marteau then outlined four political rationales and motives that could directly contribute to evidence neglect. She explained that firstly, the political cost of some policies is far too high given their unpopularity amongst the electorate, for example, taxes and purchasing restrictions. Secondly, interests and ideologies are antithetical to some effective policies. Neoliberal ideologies promoting free markets and personal responsibility often conflict with evidence-based views of unhealthy and unsustainable behaviour which are driven by environments for state intervention is needed, for example to restrict advertising. Thirdly, Marteau argued that there is an overreliance on technologies as an all-powerful climate and health silver bullet, and not enough focus placed upon shifting public behaviours which are necessary for achieving current government health and climate ambitions. Finally, she said corporate lobbying can act against forming the legislation to achieve health and climate ambitions, casting doubt on the evidence for effective ways to change behaviour at scale. Along with the petrochemical, tobacco, alcohol and meat industries, junk food companies lobby against forming impactful legislation to combat childhood obesity.
Gill progressed the themes discussed by Marteau with contextual insight into how evidence is used within government. He explained that decisions of high importance are often coupled with a higher number of associated stakeholders. Thus, he suggested that governments must balance the competing priorities of industry, constituents’ wishes, party manifestos, virtue and moral good, and academia when making a decision. Such decisions may also involve trade offs between different priorities with different time horizons (e.g. net zero and the cost of living crisis). In such contexts, the usefulness of quantitative evidence becomes diminished as it is not possible to quantify a trade off between incommensurable priorities. Dr Gill maintained that if a decision-maker is presented with many different conflicting perspectives, the main challenge then morphs into how to weigh and calibrate between these divergent directions. Dr Gill suggested a focus on how academia can help decision-makers in such contexts to act in a robust, intellectually informed way.
“Given the incentives not to use evidence, having the structural requirements to use evidence is very important.” – Dr Matthew Gill
The discussion then opened to participants; one voiced their struggles when engaging with different parts of the civil service. They explained they’d experienced a lack of understanding of the relative strengths of different types of evidence. For example, evidence generated from randomised control trials and focus groups ought not to have the same influence; respective gradation of types of evidence change and depend upon the situation at hand. Dr Gill aligned with this sentiment, he argued that we should focus upon the creation and generation of evidence, but we must equally and seriously consider how it is used, which is much harder to articulate, formalise, and legislate for. Dr Gill explained that “knowledge management” is crucial if the government hopes to meet its ambitious health, wellbeing, and climate targets.