Making Sense of Science and Evidence

Abstract

The public has to negotiate competing claims about which products, practices and policies will work. If we expect people to make sense of these, then knowledge about evidence, and an understanding of how it is obtained and reviewed, is vital. However, this public language of science has been neglected. The public is encouraged to admire science and to be entertained and amazed by it. We should be doing more to equip people with insights into evidence and scientific enquiry, and expanding discussion about what we know and how we know it.

Every day we are bombarded with stories about science and health, many of which have a direct bearing on our lives. How much alcohol is it safe to drink? Should pregnant women avoid certain foods? Is alternative medicine effective and safe? How dangerous is nuclear power? Is climate change real? Will my daughter's mobile phone give her brain cancer? What, exactly, is the link between diet, exercise, obesity and health?

Figure 1 ‘I don't know what to believe…’ public guide to peer review.

Sometimes the issue is made even more complex as findings appear to directly contradict received scientific wisdom, and even the results of the last study to hit the headlines. It can be all too easy for the public to conclude that many studies are given publicity merely for the sensation this will cause.

Figure 2 Press coverage of early career researchers' campaign to debunk detox.

So it is not surprising that a growing number of people – members of the public and organisations that represent and help them – are asking for help about a wide range of science-related subjects, to sort through scare and hype stories, to determine which products and practices are effective or what might be a scam, and to come to conclusions about the reliability of scientific claims and assess controversies.

Many of the questions that people seek help on are about establishing the reliability or ‘meaning’ of research claims: what do scientists actually know about this? Are they just guessing? How ‘hard’ is this evidence? How big was the trial? Was it properly peer-reviewed? How confident can we be that these results are meaningful? Who funded this research? Can I find out what tests have been done? How do we know?

This is the public language of research and it needs more attention than it is getting now. In the 1950s, the great populariser of science, Jacob Bronowski, talked about the dangers of losing ‘a sense of science as a social activity’ and noted that chief among its symptoms was the decline of a common language.

It is a language that emerges when we lay bare the reasoning behind different claims about science and evidence, not in the sterile and cynical way that rests at ‘who funded what’ or that satisfies itself with narrow-minded glee at pointing out limits and errors as though scientific enquiry is just a pretension or ‘matter of opinion’, but in a way that empowers us to question our assumptions and received wisdom.

It is easy to forget how bewildering the ‘science scene’ is for the uninitiated. Assuming you have no familiarity with law, imagine that you are asked to write an article about the legal implications of the community blog Mumsnet, a website that is hugely popular with young mothers, being sued for libel. And you have to do this by tomorrow afternoon.

Where do you start? What questions do you ask? Who do you ask? Think of that huge edifice that is the legal world, and just imagine trying to formulate what you are looking for, never mind find it. Even if you get as far as the existence of the Defamation Act, subsequent amendments, and what they do, how confident would you be about who to call and how much of the picture their view would be giving you?

Now consider being a jobbing writer on a parenting magazine, living outside London and wanting to write about plasticisers in babies’ bottles. Or a head teacher looking at a promotion of brain-training products to improve pupil performance or reviewing a request for permission to conduct clinical research in the school.

Many phone calls to ‘Sense About Science’ from people in these circumstances begin with ‘I'm not really sure who to ask, I'm not really sure how to ask this.’

This is unsurprising. Because despite the recent proliferation of science communication, the popularisation of scientific reasoning, a public celebration of the scientific method itself (and the tools by which people can assess the evidence for themselves), has been relatively neglected.

This seems counter intuitive because in many ways we live in a golden age of science communication. The airwaves, magazines and newspapers are full of science, environmental, technological and medical stories in a way they certainly were not even 30 years ago.

Researchers today embark on their careers with some expectation of their work being publicised, and in many countries research grants now regularly stipulate it, colleagues more often support it and institutions run outreach programmes and festivals to promote it. Universities employ press teams who actively liaise with scientists and prepare them for the deluge of interest that will inevitably accompany any major finding or publication.

Communication theory, training and governance advice have become crowded academic marketplaces, initially in the UK and parts of Europe but increasingly elsewhere. There has been a significant promotion of science as a leisure activity and cultural interest and there have been sustained efforts to make the public-interest case for investment in scientific research and development.

All great news, but the question remains, how can we better communicate the nuts-and-bolts of the scientific method itself?

There is now a distinct move towards organising some areas of science communication with reference to ‘how do we know?’ The new climate gallery of the Science Museum in London has been designed to encourage visitors to consider climate change in the context of how we expect scientific knowledge to develop, through peer review, challenge and self-correction.

The Museum's (former) director, Chris Rapley, who instigated the new gallery, has contrasted this to the less helpful framing of climate discussions as a debate in which one side has the right answer. Journalists and broadcasters are also contributing to fuller discussions about the status of scientific claims.

Soon after the creation of Sense About Science in 2002, one of the public understanding gaps that quickly became clear was the whole issue of peer-review. This seemed to be a well-kept secret of the research community. In no other area of life do people systematically volunteer their work to be critically evaluated by others in their field. Can you imagine a Government department's press releases being submitted for critical discussion to MPs before publication?

Our Working Group on Peer Review, established in 2004 and chaired by Professor Sir Brian Heap, concluded that it was a process little understood by many who interacted with the findings of research. Yet for them, and the public at large, insights into how research had been evaluated were highly valuable. The Working Group resolved that greater effort was needed by research institutions, journals, publishers and others to share the workings of these processes

So progress is being made. And yet we find that, in all societies, we have barely begun to explore popularising science as a tool for empowering civil society.

Figure 3 Supporters of Sense About Science's ‘Ask for evidence’ campaign: www.senseaboutscience.org/pages/a4e.html.

Perhaps in part it is that too little ‘science communication’ actually addresses itself to the trickier debates and controversies. Perhaps also it is the result of defensiveness. The recognition, by Walter Bodmer and others, that the relationship between science and society needed attention also occurred in the context of a loss of confidence in science.

This defensive posture has led to a focus on more anxious, defensive and self-regarding questions at the expense of seeing what scientific reasoning contributes to the public interest.

Discussions following the House of Lords report on Science and Society¹ prompted several initiatives for putting the relationship between science and the public on better terms. Sense About Science was among them. Born into the fire of debates raging about cloning, GM, ‘man-made chemicals’, the MMR vaccine, organ donation, alternative medicine, mobile phone radiation and more, Sense About Science was faced with the huge question: how can the public get a handle on the scientific evidence?

Scientists often ask ‘What are we to do when commentators or products can make sweeping, unsupported assertions but our responses must be tentative and full of caveats?’ However, it is not necessary to be as coy about tackling misconceptions as many have been in science and policy communication. When the purpose of an activity or commentary is to address misconceptions, it should do so directly, not euphemistically.

By moving away from the defensive position and starting with the public discussion, it becomes possible to give a more meaningful response. We may not know everything about the immune system, for example, but we know that the fashionable food-intolerance tests, which are heavily promoted on the Internet and by some alternative health therapists, produce arbitrary results that are not supported by evidence and defy basic physiology. We can say so. There is plenty of uncertainty concerning the likely effects of climate change yet this does not mean that we know nothing about the way the planet is warming and the effects of CO₂.

There is also much that can be done to equip people with some straightforward insights that would help them to resist the influence of misleading stories and claims. Many such stories arise, for example, when a correlation is interpreted as causation. There are three insights that enable people to unpick such stories. First, it is helpful to know that associations tend to appear by chance and often disappear when a different sample is used. Second, that the observation that two things are linked, or correlated, is the first hint of a relationship but does not show that one thing causes the other. It could be that the research is missing part of the picture, or that there are other explanations for the observed link. Thirdly, to show that A causes B, scientists look for strong and consistent evidence across a large body of research, and a plausible biological mechanism to explain the link.

Defensiveness among institutions, though, has caused scientists, universities and politicians to miss such opportunities to help people use the tools of scientific reasoning to clarify the debates in front of them. Instead they have mistaken the more cynical, relativistic reactions to science as the interest of the public, when they are no such thing. Defensiveness about public attitudes to scientific matters also causes them to cast about for the fruits of science with which to ‘wow’ people.

There is every reason to think this situation could be improved. Asking about how knowledge is acquired is a good way to learn about complex scientific issues. And the public is, perhaps surprisingly, more aware than many assume when it comes to basic assessments of logic and scientific rigour.

Surveys show that people without a background in science readily perceive questions about evidence and scientific enquiry. The Wellcome Trust Monitor found that 70% of adults correctly identified that the scientific way to test the effectiveness of a drug was to give it to some patients but not others and compare their outcomes.² As an answer this seems so obvious that it is almost banal, but it is worth remembering that until well into the Enlightenment almost no scientists or medics thought this way.

An even higher number (79%) answered questions about probability correctly. Clearly, there is an appetite (and the intellectual discernment) among members of the public for understanding how scientists and policymakers have reasoned something through. Nevertheless, it is very rare that scientists have themselves immediately identified the insights that best equip the public to make sense of claims they encounter. It is often only through a persistent review of public discussion that their different assumptions become clear.

It is true that not all findings are as optimistic about general public understanding. The Eurobarometer³ and the US National Science Foundation⁴ surveys, have put the figure for ‘scientific literacy’ at around a fifth of the population. Putting aside differences in methodology and assessment, and the extent to which the surveys tested prior knowledge, it is worth noting that a fifth of the adult population being equipped to discuss the nature of a scientifically-designed test is far from disastrous, nor does it suggest a limit to the development of a sceptical and engaged civil society.

The many questions that people have about the scientific basis for assertions are testament to the public's desire to find a way into discussions from climate change to mercury in vaccines. As the author Thomas Berger is reported to have said, ‘the art and science of asking questions is the source of all knowledge’. We can respond to this only by rediscovering science as a public tool for testing the plausibility of competing claims, and by encouraging a sceptical and inquiring public that readily asks ‘How do we know?’