Library Element Report

Making sense of science for policy under conditions of complexity & uncertainty | Evidence Review Report

Uploaded by RRI Tools on 10 July 2019

Making sense of science for policy under conditions of complexity & uncertainty | Evidence Review Report by SAPEA. July 2019.

The need for science advice

1. Science  advice to today’s policymakers has become more prominent than ever, due primarily to the growing human impact on our world, and the everincreasing complexity of the knowledge needed for coping with economic, social and environmental challenges. These include demographic changes, global trade issues, international market structures, transboundary pollution, digitalisation, urbanisation and many other factors of modern life. Many such policy problems are characterised by a mixture of complexity, uncertainty and ambiguity. The issues for which scientific input is most needed by policymakers are the ones for which the science is most often complex, multidisciplinary and incomplete.

2. Scientific expertise supports effective policymaking by providing the best available knowledge, which can then be used to understand a specific problem, generate and evaluate policy options, and provide meaning to the discussion around critical topics within society. Scientific knowledge is crucial to ensuring that systematic evidence is part of the collective decisionmaking process. Systematic knowledge is instrumental to understanding phenomena, providing insights that help to understand and tackle society’s problems. Science therefore represents an essential element in Europe’s future development of policy.

3. The nature of science advice is wide-ranging. The science advisory ecosystem includes a broad set of players, from individual academics to national academies, universities, think tanks and many others. Their roles include knowledge generation, synthesis, brokering, policy evaluation, horizon scanning and more.

4. In the vast majority of policy cases, scientific advice is only one of many inputs but it occupies a unique position, as summarised below and in the report.

The purpose of the report

5. This SAPEA Evidence Review Report draws on the work of a group of European experts from many disciplines and many countries. Our Evidence Review Report supports the work of the European Group of Chief Scientific Advisors (GCSA), which has published a Scientific Opinion on Making sense of science for policy under conditions of complexity and uncertainty (Scientific Executive Summary 12 Advice Mechanism, 2019). The GCSA’s Scientific Opinion is addressed primarily to policymakers who utilise scientific advice across the European Commission, and is also of relevance to the governance of scientific advice in the Commission. The overarching question posed by the GCSA is: 

How to provide good science advice to European Commission policymakers, based on available evidence, under conditions of scientific complexity and uncertainty?

The structure of the report

6. This report has four main chapters. In Chapter 2, the report introduces some of the key terms that are the building blocks for the rest of the report. Chapter 3 explores the prospects, limitations and constraints of scientific advice for policymaking. Chapter 4 illuminates the policymakers’ needs for scientific input and advice. Chapter 5 addresses the potential for enhancing the interface between scientific evidence and policymaking. The report concludes with a summary of the main findings, in the form of lessons learned.

The debate about science

7. There are many schools of thought in the study of science and science advice that provide quite distinctive answers to the question of what science is or should be, and how it can be best used in policymaking. These come from the philosophy of science, the sociology of science, the study of knowledge and many other traditions in the field.

8. Members of the SAPEA expert group are motivated to overcome traditional schisms between different schools of thought in this area. The report is based on a pragmatic, consensus-oriented interpretation of the published literature and familiarity with theory and practice. The report includes, wherever possible, evidence from the empirical study of advice processes, but it also refers to and cites interpretations and conceptual thoughts of many scholars devoted to studying the nexus between science and policy.

9. Most concepts of science agree that its purpose is to produce and test claims about reality. It includes statements that are descriptive (how reality is shaped), analytic (causal and functional relationships between phenomena) and, depending on the specific discipline, normative (how reality should be changed or altered).

10. Systematic knowledge is generally generated and evaluated according to established rules and conventions of the respective academic discipline. These rules are not perfect, yet they are regarded as superior to any other alternative.

The role of science in policymaking

11. Scientific experts provide knowledge that helps to provide evidence to the policymaking process. ‘Evidence’ can be defined as a knowledge claim that is backed up by a recognised scientific procedure or method.

12. Scientific knowledge and understanding represent an essential dimension of many policy decisions. The contributions of scientists to policymaking should be encouraged and valued. Scientific advice is neither arbitrary, nor is it a direct representation of an objective ‘truth’. At its best, it is based on methodological rigour, agreed-upon rules of enquiry, systematic review of evidence, and continuous analysis and debate.

13. Science advice must be based on the best available evidence, communicated in a transparent and accountable way that explicitly and honestly assesses uncertainties, ambiguities and tensions.

14. In the current climate of populism, ‘post-truth’ and ‘fake news’, public scrutiny and the accountability of science are an inevitable and even desirable aspect of democratic processes.

Bringing science advice to policy

15. Methodological rigour that seeks to attain valid, reliable and robust evidence remains the most important means of judging the quality of scientific insights.

16. Many policy options require systematic knowledge that is not available, or still in its infancy, or in an intermediate state. There may be an incomplete understanding of the phenomenon and no clear causal relationship; it may rely on educated guesses by experts. These all need to be labelled as such and it is essential to demarcate the limits of ‘reasonable’ claims.

17. Making sense of science cannot be done by only looking at the empirical evidence. On the contrary, it requires lots of room for interpretation and inter-subjective judgement. The question of what counts as the ‘success’ or ‘failure’ of scientific advice for policymaking cannot be determined by objective measurements. There are many valuable indicators, but interpreting such descriptive data requires experiential knowledge (i.e. familiarity with the
topic) and prudent judgement.

18. Scientific outputs often represent the best available systematic knowledge on a given subject, but this is not the only relevant or necessary knowledge that decision-makers should use. Knowledge based on actual experience and local perspectives can often be provided only by people, who share common experiences with the policy issue under consideration. The term ‘evidenceinformed’, rather than ‘evidence-based’, therefore assures that all evidence is considered but does not become the sole basis for decision-making.

19. What counts as ‘good’ evidence varies with the questions: it depends on what policymakers want to know, for what purpose, and to what context the scientific advice is being addressed. Most practices on the presentation of evidence and its appraisal are focused on social values of legitimacy, trust, impartiality and credibility.

20. Complexity is a major barrier to providing reliable insights about the likely consequences of decision options. Examples of highly complex phenomena include climate change and economic crises. Individual scientists may not be able to see the entire picture, but experts from different disciplinary perspectives can give policymakers a more complete picture of what science knows and does not know, and about the robustness of available evidence.

21. In the context of decision-making, uncertainty relates to a situation with more than one outcome consistent with expectations. Scientific uncertainty relates to the limitedness or even absence of scientific knowledge (i.e. data and information) that makes it difficult to assess the exact probability and possible outcomes of unwanted effects. Uncertainty management and quality assurance are essential in any decision-making process. Scientific
uncertainty can be communicated effectively by characterising, assessing and conveying the limits of scientific statements clearly. In particular, it is necessary to ensure that policymakers understand the meaning of probability distributions, confidence intervals and statistical quality criteria when interpreting uncertainty characterisations and are well-informed about the assumptions and conventions that are incorporated in various scientific assessments.

22. While more and better data and information may reduce scientific uncertainty, more knowledge does not necessarily reduce ambiguity. This report uses the term ‘ambiguity’ to represent the plurality of scientifically justifiable viewpoints on the meaning and implications of scientific evidence.

Conclusions of the report

23. The conclusions in the report are the results of a creative process of combining empirical evidence, positions from the literature and personal reflections by those who have been active in giving scientific advice for many years. These are:

  • Science advice can help to anticipate future challenges and assist in designing coping strategies or interventions in a world in which human actions have become the dominant force in shaping it (the Anthropocene era).
  • The focus of science advice must be on a critical review of the available evidence and its implications for policymaking. It is important that scientific advice is based on evidence that is respected as valid, relevant, reliable and (depending on the academic discipline involved) replicable. It should include a quantitative assessment or, if that is not possible or feasible, a qualitative characterisation of scientific uncertainty and ambiguity. Some of the EU agencies have made progress in this area, and it would be a welcome initiative if guidance and best practice were shared.
  • Scientific advice should not prescribe but inform policies. Any political decision needs to consider the likely consequences of decision options (where scientific input is essential) as well as the social, political and moral desirability of these consequences (where plural values and ethical principles play a major role). In the end, any scientific advice may turn out to be incomplete, contested or even unsubstantiated. The selection and interpretation of evidence must be guided by the articulation of different social values and legitimate interests, involving not only advisers and decision-makers, but also additional stakeholders and civil society.
  • The purpose and significance of scientific advice depend on the issue and the context. There are many forms and sources of knowledge. Science advisers should see their role as important, and also as a unique source of robust and reliable knowledge, but not as the exclusive providers of knowledge. When policymakers and science advisers agree in advance on the role and function that scientific evidence should play, it should lead to greater clarity and collaboration.
  • Form and function are vital when designing appropriate policy-science interfaces. There is no universally applicable model for structuring scientific advice for policymaking. The type or nature of available expertise and the type of advice needed should determine the procedure, structure and composition of the advising process.
  • Science advice for policymaking involves many legitimate perspectives and insights. Defining ‘the issue’ and selecting the most appropriate expertise requires judgement and vision. For complex problems and issues, it is essential that the complete range of scientific opinions is represented and that all uncertainties and ambiguities are fully disclosed.
  • Scientists, as well as policymakers, should be sensitive to various biases and interests when drawing inferences from data and information. Having access to different disciplinary perspectives (for example, the humanities, natural sciences etc.) can act as a check and balance procedure to address unintended bias.
  • Science advice is always affected by values, conventions and preferences. Rather than highlighting the role of the ‘objective’ knowledge provider, the science-policy nexus is better served when both sides are transparent about what values and goals they apply and how knowledge claims are selected, processed and interpreted. This creates more trust and confidence in institutions and in the processes for science advice.
  • The effectiveness of scientific advice depends on the right composition of advisers and the quality of the dialogue between advisers and policymakers. Science advice should include evidence that clarifies and explains the factual content of an issue, including a characterisation of its robustness and validity, together with the ethical and societal impacts of the topic and the values involved. When translating evidence and research findings, issues such as transparency, openness, assumptions and uncertainties must be addressed and communicated. Advisers should accept some level of responsibility in advising and in the implementation phase of their advice. Feedback on the effects of the advice is needed, which can be used for adjustments or correcting actions during its implementation.
  • The relationship between science advisers and policymakers relies on mutual trust. It is important to maintain a capacity for reflection, as well as openness on the part of policymakers to disruptive advice.
  • The most highly recommended science advice process combines analytic rigour with deliberative argumentation. Analysis refers to the inclusion of systematic and peer-reviewed knowledge. Deliberation refers to the mutual exchange of arguments and reflections, to arrive at evidence-informed and value-balanced conclusions in a discussion.
  • Stakeholders and citizens should be integrated into the process. Continuous forums for deliberations between the scientists, the public and policymakers should be fostered. Critical elements to be considered include the transparency of aims, the means of power regulation between the different stakeholders, and responsive communication strategies.
  • Science advice is not limited to policymakers but includes science communication to the wider society. Effective science communication includes clarity about the quality of evidence, the treatment of uncertainties and ambiguities, the possible courses of action and information about the background of the science advisers themselves. Effective partnerships between scientists, policymakers and practitioners (who implement policy decisions) will help to build trust and credibility.

Full report
Executive summary



Related Resources