Policy making under pressure

Policy making under pressure

Deniz Gursul, the Campaigns and Policy Manager for the Royal Statistical Society (RSS), joined the CSaP online seminar series on ‘Data for Policy Making: Challenges and Opportunities’ to present on the ongoing project 'Statistics Under Pressure'. The initiative aims to address the challenges of using statistics in policy making under pressure, particularly evident during the Covid-19 pandemic.

The RSS is a professional body and charity representing statisticians and data scientists that advocates for the importance of statistics and data in decision-making for the public good. The RSS provides various resources, professional development opportunities, and networking platforms for its members.

'Statistics Under Pressure' was initiated in response to the pressing need for timely and reliable data during the Covid-19 pandemic. Decision-makers faced challenges in determining when data was sufficient to inform decisions and navigating trade-offs in the absence of perfect data. The project aims to foster an environment where data, even if imperfect, can confidently inform time-sensitive decisions. Ms Gursul outlined five primary objectives of the project.

1. Deepening understanding among statisticians on trade-offs

The project aims to educate professionals on when and how to make suitable trade-offs when dealing with time-sensitive data analysis and decision-making. It provides guidelines and best practices for navigating the tension between data quality and timeliness, ensuring that decisions are based on the best available information under pressure.

2. Improving non-analysts', such as policymakers and stakeholders, ability to interpret uncertain data

This objective recognises that decision-makers often rely on statistical information to inform policies and actions, but may lack the technical expertise to fully understand the nuances associated with uncertain data. Therefore, the project will develop resources and guidance to help non-analysts interpret statistical information accurately, highlighting its limitations and uncertainties.

3. Strengthening resources for rapid data production

This consists of identifying and addressing bottlenecks in data collection, analysis, and dissemination processes to ensure timely access to relevant information for decision-makers. It may include developing tools, platforms, and protocols to streamline data production and make it more responsive to urgent needs.

4. Enhancing cross-team communication

This includes fostering better communication between statisticians and decision-makers, as well as improving collaboration within and across organisations. By breaking down silos and promoting knowledge exchange, the project facilitates more effective use of data in policy making.

5. Raising awareness about the importance of making trade-offs

Finally, the project will raise awareness about trade-offs by educating various stakeholders, including policymakers, the public, and academia, about the inherent uncertainties and limitations associated with data-driven processes, especially in high-pressure situations. By promoting a better understanding of trade-offs, the project seeks to foster a culture of evidence-based decision-making that takes into account both the strengths and weaknesses of available data.

Making decisions under pressure

Ms Gursul provided an example related to the Covid-19 infection survey to illustrate the concept of trade-offs and decision-making under pressure. At the onset of the Covid-19 pandemic, there was an urgent need to determine infection rates to inform the pandemic response. However, traditional data collection methods were not feasible due to time constraints. The team decided to utilise an existing database of participants who had previously indicated interest in research studies. While this approach facilitated rapid data collection, it resulted in a sample that was not as representative as desired. Participants in the database may have had different characteristics, such as higher rates of vaccination or education, which could bias the results.

The team had to balance the need for timely results with the desire for a representative sample. By prioritising speed, they accepted the trade-off—reduced representativeness in the initial stages of data collection. Additionally, the team deferred detailed subgroup analysis to focus on core information needed for immediate decision-making. However, this approach faced criticism from the statistical community due to concerns about sample representativeness. The team considered alternative methods that could have yielded more representative results, however, such approaches would have taken longer to implement, rendering them less suitable for informing immediate decisions.

Currently, the project is focussing on developing case studies to illustrate trade-offs made in real-world scenarios, such as Covid-19 data collection and transportation statistics. These case studies aim to inform the development of principles that guide decision-making under pressure, ensuring data is fit for purpose whilst acknowledging its limitations. Ms Gursul highlighted the importance of clear communication, collaboration, risk mitigation, and prioritisation in decision-making under pressure. She emphasised the need for a minimum viable product to inform decisions promptly while allowing for refinement over time.

Image by Pietro Jeng on Unsplash.