News

Government data, science and evidence: national security

10 November 2021

Share

Government data, science and evidence: national security

Reported by Ryan Francis, CSaP Policy Intern

In CSaP’s first Policy Fellow seminar on the government's use of data, science and evidence we welcomed Suzanne Raine, Affiliate Lecturer at the Centre for Geopolitics, University of Cambridge. Suzanne asked the question: “What is the role of evidence in risk management, anticipation and warning?” and drew on her experience leading the Joint Terrorism Analysis Centre, looking at wider lessons for governments.

National security and government

In issues of national security it is the job of the National Security Council to anticipate a threat and decide on the appropriate action. More widely, however, it is the duty of government to protect its citizens, institutions, and economy against all possible risks and hazards, such as those listed in the National Risk Register. This includes many different types of threats, for example: terrorist attacks, hostile state activity, cyber-attacks but also pandemics, economic volatility, access to resources, and natural hazards, such as wildfires and floods, volcanic activity, earthquakes, severe storms and hurricanes.

Communicating warnings

We must accept that events will happen at some point in the future, even those events at the high impact – low probability end.

Raine focused on the question of how best to produce a warning and communicate it effectively. Throughout the seminar, participants discussed how best to anticipate hazards and take decisive action to avoid impending threats. Currently, the UK government uses warning systems to produce probabilistic threat levels against some risks and threats. For instance, as we saw during the COVID-19 pandemic, there were different threat levels introduced to advise the public on whether it was safe to travel to certain countries, which corresponded to the likeliness of someone catching the virus based on the infection data from a certain country.

Raine said we must accept that events will happen at some point in the future, even those at the low probability, high impact end of the risk register. She suggested that we cannot anticipate risks and respond without a “coherent risk management system” in place, and that we cannot rely on evidence alone: there is a “logical gap”, because somehow evidence has to be transformed into a projection of likely future consequences of a course of action, if it is to enable mitigation of a risk.

The role of evidence in government decision-making

Throughout the seminar, Raine explained that warning systems must be designed to make use of all available information, recognising that in many cases this may not be the complete picture and may fall far short of what we would think of as ‘evidence’. Thus, to build the best warning systems, we must perform critical epistemic analyses of the role evidence plays in government decision-making. The following questions were then raised: should decisions be made before the evidence is clear? What are the limitations of evidence: is evidence always definitive? Is past evidence always applicable to future events? Raine argued that these questions highlighted the insufficiencies of evidence-based decision-making. The participants then highlighted that, even though evidence-based policy is current and well-established, effective decision-making must extend far beyond just considering the data.

Analysing the evidence

Raine explained that in almost every circumstance, we have an “infinite amount of information available” which she branded as “useless” unless it has been filtered into a form that can be understood, such that conclusions can be drawn which enable the right risk response.

She maintained that the Government’s warnings needed to be clear, authoritative and penetrate thought ambient noise: as primarily these warnings must be heeded by the public and institutions. This required agreement across the system, on the point at which consensus was required and where responsibility for warning and action could be clearly devolved to individuals or individual organisations. This organisational design is similar to that of the private sector, where organisations often manage risks through a structure of risk owners.

The importance of humans

To round off the seminar, Raine highlighted that ultimately human beings are needed to turn evidence and data into understanding and warnings, in particular to interrogate machine-produced probabilities and sometimes to overrule them. There will be different amounts of data and ‘evidence’ for different threats and hazards, for instance, for counter terrorism there might only be a small amount of information while in contrast weather systems are being monitored constantly. The role of the human is particularly important with a novel threat or when data is clearly insufficient.

Secondly and crucially, she asserts that what evidence cannot reveal is when to act. A warning after the threat has materialised is not a warning, even if the evidence is by then perfect. Raine explained how deciding when to warn is a “really difficult nail-biting process” which has to be free of any political influence or agendas. Her overarching argument was that humans play a ‘critical’ role in analysing evidence and data, because they possess wisdom, experience, courage, and intuition that computers do not have.

Human beings are needed to turn evidence into understanding and warnings.

Ultimately, this hybrid (human-computational) approach to public policy invariably introduces many challenges; and questions remain on how we guard against human biases. For example, it was raised that most commonly, recency biases often divert evidence off-track and on to what humans “believe”, as opposed to what the information or evidence objectively shows. Raine suggested that this is not an argument against investing in human analysis, but instead an argument to invest in training the analysts to reduce likelihood of analytical failure. There is ongoing scientific research into the effects of human biases upon decision-making in governments.


Ryan Francis

Centre for Science and Policy, University of Cambridge

Suzanne Raine

Department of Politics and International Studies (POLIS), University of Cambridge