Share
Reported by Nick Cosstick, AHRC-funded policy intern (September 2017- December 2017)
A public panel discussion, organised in collaboration with the Winton Centre for Risk and Evidence Communication in Cambridge last week, approached the theme of communicating risk from the perspective of the distinguished panellists. A major focus in each of their talks was the relationship between the communication of uncertainty and the general public's trust of experts.
View the talk here:
How can we communicate uncertainty without endangering trust and credibility?
We are at a stage where the level of distrust in experts is more perceptible than ever before. One causal factor for this situation is the lack of a reliable methodology for communicating uncertainty about the conclusions we draw from our data. Even our best scientific methods do not lead us to utter certainty; yet so often in the public communication of research, uncertain conclusions are presented as certainties. When such conclusions fail to be 100% accurate, as is inevitable with uncertain conclusions, the public trust in – and credibility of – experts is damaged.
Sir David Spiegelhalter, Winton Professor of the Public Understanding of Risk in Faculty of Mathematics at the University of Cambridge, tackled this issue head on in his talk. He outlined multiple facts that we can be uncertain about, highlighting: facts about the present and past (e.g. current unemployment rates), facts inferred from mathematical, statistical or computational modelling (e.g. the impact of a particular policy), and scientific facts (e.g. that a particular scientific theory is an accurate representation of reality). He then outlined the broad range of ways that we could communicate uncertainty, from giving a full probability distribution to verbally qualifying that we are uncertain.
Sir David outlined numerous examples of ways in which government departments, NGOs and academic institutions have managed to communicate the conceptually tricky notion of risk without horribly complicating the presentation. For example, the British Medical Journal compares interventions via a comparison of their benefits and harms, and an assessment of the quality of evidence upon which the comparison was made, on a 1 to 4 star scale. Sir David described the summary of a complicated set of evidence into a single star rating as “a really powerful idea”.
Finally, he highlighted that the question of how we communicate uncertainty without endangering trust and credibility is a research question that academics at the Winton Centre are investigating. Sir David did have some broad predictions. He argued that the answer to this research question is likely to depend upon the context of the communication, its audience, and the format of presentation. However, since the research programme is currently in its infancy, this is not certain.
Finding the story behind the numbers
According to the 2016 BBC Impartiality Review – "audiences appear to put a premium on statistics and information which can illuminate a confusing situation and add some accurate knowledge". Achieving this aim, whilst also keeping the information as clear and simple as possible, is a tremendous challenge. Amanda Farnsworth, grapples with this challenge every day in her role as Head of Visual and Data Journalism at BBC News.
Her talk outlined the BBC's strategy for communicating data. She stressed the importance of using graphics rather than pure numbers to communicate the message. Any pure numbers that are used are spaced out appropriately from other numbers, so that the audience can focus on the pertinent information. Most importantly, any big numbers need to be scaled down and humanised. Amanda gave the hypothetical example of communicating that the government plans to give an extra £5 billion to child care for the under 4s over the next five years. Such a statement does not allow the audience to know exactly how big an impact that will make to their lives. Were it to be qualified by the information that this policy amounts to an extra £100 per child over that period, they would have a far greater understanding of its relevance.
In order to communicate uncertainty, the BBC has made changes to its visual representations of data. For example, the BBC's 2017 general election poll tracker, represented the result of every opinion poll as an individual data point - including outliers - as well as the curve fitted to the data. It also did not include a percentage share summary of the polls. A key focus for the BBC is to set rigid standards regarding its use of language. This means not only qualifying represented data as the "best estimate", but also communicating clearly what is meant by terms such as "likely".
Amanda finished by highlighting the impact that technology has had upon the consumption of news. A salient phrase in the epoch of social media is – "we are all experts now". Like it or not, the general public gets much of its news from social media. Sometimes the information shared in such networks is of high evidential quality, at other times it is evidentially poor, even outright fake. She stressed that the traditional media needs to develop strategies for inserting themselves into the centre of these social media networks, as trusted sources.
Uncertainty as an integral part of policymaking
The role of a policy maker is to understand which policies are best supported by the evidence, to communicate this, and to effect the policies decided upon by ministers. Recollecting the Archbishop of York John Habgood's turn of phrase "certainty is a sin", Emily Miles, Group Director of Strategy at the Department for Environment Food and Rural Affairs, argued that uncertainty is at the heart of policymaking. She called upon three cases to make her argument.
The first case was Tony Blair's commitment to halving asylum within six months. The commitment, made without any real consultation of Emily – who at the time was his asylum intake adviser – seemed altogether naive. The view at the time was that it was almost certainly doomed to failure. However, with a concerted effort around policy, delivery and tracking, the target was achieved. Emily argued that this case exemplifies the "beautiful tension" between the vision of politics and the reality of evidence. We need visionary politicians to set bold targets, as their ambition has the power to shift bureaucracy. We also need policymakers to be steely-eyed realists regarding what the evidence is telling us. The tradeoff between these two outlooks creates a situation of uncertainty – it is uncertain whether a particular policy is too bold, or not bold enough.
The second case was Avon and Somerset Police's target to reduce burglary rates. The uncertainty here is a definitional one. Avon and Somerset Police would give more resources to burglary investigations, however the National Crime Record Standard's definition of burglary is very strict, and not identical to the general public's definition. Therefore, which crimes do, and do not, get additional resources depends on categories decided upon by a body which does not categorise in the same way as the general public. It is uncertain to what extent polices which utilise these categories are truly in line with the needs and wishes of the general public.
The third case was the Environment Agency's communication of evidence to the general public during the 2014 flooding in Somerset. They hadn’t appreciated that they were not a trusted interlocutor locally. The local community didn't accept that the cited dredging research was accurate. Emily's point was that the facts are extremely important, but there also needs to be trust for the disseminator. This is especially true in cases in which the best evidence we have is uncertain. Recently, the Environmental Agency has been doing good work researching the communication of flood risk.
Can the mistrust in the use of statistics by the government be transformed?
According to polling from 2016, 78% of people agree that official statistics are generally accurate. On the same poll, however, only 26% of people surveyed agreed that the Government presents official statistics honestly when talking about its policies. This shows there is an enormous gulf between the general public trust of official statistics, and their trust of the way the government uses these statistics in policymaking. Sir David Norgrove, Chair at the UK Statistics Authority, questioned the extent to which his organisation could bridge this gap.
The role of the UK Statistics Authority is the "promoting and safeguarding the production and publication of official statistics, that 'serve the public good'". It includes a production arm - the Office for National Statistics - and a regulatory arm - the Office for Statistics Regulation. The Office for National Statistics is looking to become more outward looking in its approach. This will involve horizon scanning for the types of evidence that policymakers need to effect change. The Office for Statistics Regulation, on top of designing the Code of Practice for statistics, is also convening groups of experts to assess the coverage, completeness and usefulness of statistics in particular areas. One goal in this regard is to attempt to make it more difficult for politicians and the media to misuse statistics.
Sir David famously intervened in the Brexit debate, sending a letter to Boris Johnson regarding his misuse of statistics. This action represents the pinnacle on the hierarchy of the ways in which the UK Statistics Authority intervene to prevent the misuse of statistics. However, Sir David argued that cases in which politicians truly misuse statistics occur very infrequently. If that is the case, why does the general public mistrust the government’s use of statistics? Sir David's opinion is that the general public's attitude reflects a more general mistrust of politicians. Events such as the expense scandal and the recent sexual harassment scandal undermine the hope of regaining the public's trust.
Banner image by Susanne Nilsson licensed under CC BY 4.0