RRI Tools

To reach this goal governance structures have to be flexible, adaptable to local contexts and changes. They must have clear aims, feasible responsibilities for all actors involved, and expected impacts and follow-up. What matters in the end is how governance structures fit specific conditions adapted to the nature of your organisation, how procedures and guidelines are developed and adopted, and how they are perceived by actors in terms of legitimacy.

Steps to organising anticipatory, reflective, deliberative and responsive R&I

It may seem trivial, but the most important conditions for successful RRI implementation in any organisation are the general conviction that RRI practices will improve the organisation’s functioning and the willingness of all to act according to RRI principles. RRI implementation sometimes requires comprehensive transitions, sometimes only a conscious collective rethinking of existing practices. But it always requires dedicated time, energy and resources. Useful recommendations largely depend on the type of organisation. Organisations should reflect on the meaning of RRI, assessing its potential impact and how it could contribute to improving their activities. The stepwise Co-construction Method of the Responsibility Navigator and the Self-Reflection Tool of the RRI Toolkit can help. Specific recommendations and examples to guide you through the process are discussed below.

1. Anchor RRI in governing structures

   Socially and environmentally responsible organisations increasingly formulate and apply policies for ‘good governance’. This entails a commitment to (research) integrity, accountability, transparency, and respect for and engagement of stakeholders. It may mean applying RRI aspects in the chain of accountability, beyond the idea of corporate social responsibility, such as:

   • engaging stakeholders in the governing board or the advisory council. Valuable insights can be found in AccountAbility and in Global Corporate Governance Forum’s guide Stakeholder Engagement and the Board: Integrating Best Governance Practices.

   • making the composition of governing structures a consciously diverse representation in terms of gender and ethnicity, as described in the guides Achieving Gender Balance at the Top of Scientific Research and Diversity in Nonprofit Board Governance. More information can be also found in the Promoting Gender Balance in Decision Making section of this Toolkit.

   • setting up an ethics committee (e.g., Norwegian National Research Ethics Committees), an integrity audit committee, or a community advisory board (e.g., European AIDS Treatment Group). For more information, check out the Facilitating Structures for Reflection section.

   • establishing different hard and soft governance structures, ranging from charters, declarations and codes of conducts (e.g., research integrity, gender policy, biosecurity and dual use, nanotechnology) to principles, guidelines for structural change and checklists to institutionalise RRI processes and anchor them in work routines. More details can be found in the section How to Incorporate RRI in Policy/Funding Institutions.

2. Articulate RRI in vision and mission

   Review the mission and vision statements of your organisation from an RRI perspective. Do the statements reflect any accountability regarding gender, transparency, integrity, open access or engagement? Illustrative examples of RRI-inspired mission statements can be found at BASF, a leading chemical company that explicitly commits to product stewardship, and at Unilever, which strives to ‘make sustainable living commonplace’. The Toolkit sections How to Promote Research Integrity, Promoting Gender Balance in Decision Making and How to Implement OA Policies at Institutions provide specific guidance for each policy agenda.

3. Embed RRI in executive structures

   Dedicating human resources and assigning clear responsibilities within an organisation are required to develop appropriate structures for implementing RRI processes and instruments (e.g., codes of conduct, foresight processes, research agendas, etc.) and giving support to other professionals related to R&I. Once developed the new codes, agendas, and such must be used to adjust existing processes. In some situations ad hoc or permanent groups (e.g., research ethics committees, advisory and evaluation boards, community advisory boards) are needed to supervise, offer assessment, and support RRI processes so that a single member of the executive body is not solely responsible for the implementation and evaluation of RRI.

4. Train research stakeholders
   Incorporating RRI values within research systems needs a deep implication of education institutions and national governing structures. For this reason, some European countries have started to reformulate scientist training and career, from undergraduate to postgraduate and senior levels, to implement RRI mindsets. An examples of such initiative is the UK Concordat to Support the Career Development of Researchers, an agreement between funders and employers of research staff to improve the employment and support for researchers and research careers in UK higher education. The Toolkit sections How to incorporate RRI in higher education institutions and Training on RRI provide specific guidance and resources to train research stakeholders on RRI values.

Structures for promoting RRI in different stakeholder groups

There are numerous hybrid fora and organisations for managing multi-stakeholder processes as well as negotiation of interests, deliberation of values, and co-production of services and products. Below are institutions and resources that may help stakeholders participate, deliberate, and anticipate research and innovation.

• Policy making: (Parliamentary) Technology Assessment (TA) institutes increase the capacity and enhance the institutional foundation for knowledge-based policy making on issues involving science, technology and innovation. Examples are the Rathenau Institute in the Netherlands, the Danish Board of Technology, the PACITA project’s online TA Portal and the EST Frame consortium of TA institutes. Some cross-party think tanks involved in agenda setting constitute good examples of thought-provoking organisations that aim to bring policy, science and citizens closer. A few examples are the UK-based DEMOS, the Dutch Scientific Council for Government Policy (WRR), and the French Observatory for Responsible Innovation at Mines ParisTech.

• Research Community: Science and research organisations can organise deliberative and reflexive learning communities with peers or transdisciplinary learning communities with stakeholders. In particular, the idea of co-creation is gaining momentum, ranging from structural incorporation of learning communities, transdisciplinary research teams and living labs to community-campus partnerships. Some inspiring examples of RRI implementation are the Dutch research council NWO's responsible innovation track, the UK-based EPSRC's Framework for RRI, Farinn's facilitation of RRI in south-eastern European countries or the French National Institute for Agricultural Research’s (INRA) focus on a specific domain. Some on-going projects are currently developing Actions Plans to redefine the governance settings of some European research institutions that made them more permeable to RRI values. STARBIOS2 will generate 9 Action Plans in Bioscience research institutions over the world: 6 in European countries (Italy, United Kingdom, Poland, Slovenia, Bulgaria, and Germany), and 3 in non-European entities (USA, Brazil and South Africa. JERRI will develop Action Plans to incorporate the RRI paradigm in 2 European RTOs (the German Fraunhofer Gesellschaft and the Netherlands Organization for Applied Scientific Research). More specific resources and examples can be found in the How to Incorporate RRI in Policy/Funding Institutions section of this Toolkit.

• Education Community: It is vitally important to mainstream RRI as part of the R&I curriculum throughout the entire education system. Again, this requires dedicated human resources who embed RRI aspects and adapt existing curricula. EC-funded projects, such as HEIRRI and EnRRICH, are working in this direction. A different strategy is followed by Science Shops and comparable organisations, which actively link citizens’ needs to science, involving citizens in the research agenda setting. Valuable suggestions for fostering school-industry collaboration and ways to improve science, technology, engineering and math (STEM) education can be found at the online inGenious platform. For more specific information check out the How to Incorporate RRI in Higher Education Institutions section of this Toolkit.

• Civil Society: Many examples of collaboration with and active participation of civil society are discussed in the section How to Advocate your Idea and Set up a Project Proposal: Capacity Building for CSOs. Other interesting processes are collective awareness platforms (such as Marlisco), which raise awareness and responsibility regarding issues that affect us all as neighbours and citizens. These platforms encourage people to deal with issues such as coordinating regulation, pricing, reimbursement, recyclability, certification, safety issues, risk management and dataset practices. In Europe, Germany is pioneering with civil society organisations like GEWISS, which promotes citizen science, and Forschungswende, which created a platform of CSOs to enhance organisation capacity and increase transparency and transdisciplinary participation in the R&I system. FoTRRIS is an on-going project that aims at generating a series of CO-RRI hubs that offer efficient and effective methods for researchers, citizens, businesses and policy-makers to solve ‘glocal’ challenges, facilitating the dialogue of groups of people interested in co-solving a glocal challenge. 

• Industry and Business: The KARIM project’s Responsible Innovation Criteria stresses the competitive advantages to small and medium-sized enterprises (SMEs) from incorporating RRI principles in their mission and processes. SMEs may shift to a new paradigm of RRI or just make small changes in existing Corporate Social Responsibility arrangements. Interesting RRI-related co-creation projects, like living labs, stem from industry but are currently being adopted by government and research centres because of their successful and dynamic formulas. For more guidance check out the section How to Embed the RRI Principles in a Business Plan.

A case in point: Portugal’s Laboratórios de Participação Pública

In January 2016, the Portuguese government launched a nationwide plan to involve the public in setting the R&I agenda: Laboratórios de Participação Pública (Public Participation Laboratories). It will lead to an additional “national participatory budget” of several million euros in 2017 for an R&I programme that is voted on and scrutinized by the citizens. For this programme, the government is making use of the network of Ciência Viva science centres, which works at arm’s-length with the Ministry for Science, Technology and Higher Education.

The programme was launched in Bragança, in the Trás-os-Montes sub-region, by Minister Manuel Heitor, who deliberately chose this rather isolated part of Portugal for the launching. Local people and municipal authorities have proposed ideas for research projects their region needed, and the Polytechnic Institute of Bragança has been assigned as research partner, emphasizing the variety of Portuguese universities that could participate. The full programme of meetings is still underway, but projects specifically geared to mountain regions and to international links (particularly with Spain, which borders Trás-os-Montes) are being put forward.

Although initiated by the government, the Laboratórios de Participação Pública have “no defined rules of engagement”. This allows bottom-up participation by ordinary citizens in defining and prioritizing research agendas alongside their local representatives. The laboratórios take advantage of public engagement facilities such as science centres to provide neutral territory that is closer to the wider public than are higher education or research institutions. The programme as a whole makes use of both “open” laboratories -public participation assemblies- and more traditional focus groups, depending on agenda- and budget-setting activities and the stage of research. At present, the programme is being rolled out on a trial basis, so the available budget is limited, but future developments may see that increase.

Though this initiative comes from Portugal, it has the potential to be widely applicable across Europe. For example, it has many similarities with Sweden’s Challenge Driven Innovation scheme, one of the showcases available on this Toolkit.

Making the case for RRI

RRI as an overarching framework is a new concept, even though it draws upon more established agendas and processes. Thus, governments and institutions may need to be convinced on the benefits of adopting RRI principles and practices. It is important that policy and funding institutions understand what RRI involves and that senior ministerial and civil service teams appreciate its importance and can act as its advocates.

The RRI Toolkit offers some helpful information for understanding and implementing RRI. The European Commission’s leaflet Europe’s Ability to Respond to Societal Challenges and video Aligning R&I with European Society explain what the EC understands RRI to mean. Further insights are available in this Toolkit’s What is RRI? page and in the Project Brief, Training Showcases, Catalogue of Good RRI Practices and the Report on the Analysis of the Opportunities, Obstacles and Needs of the Stakeholder Groups in RRI Practices in Europe produced by RRI Tools.

At the national level, a good working knowledge of approaches taken across Europe can be found in the Res-AGora project’s case studies and comprehensive databases: RRI Trends (national policies on RRI) and RRI Trends 2nd Wave (RRI in companies, universities and research funding organizations). The MASIS report provides a useful snapshot (from 2012) of the state of science-and-society relations across Europe.

A variety of measures foster and support RRI at the institutional level. A straightforward background of RRI is outlined in Euroscientist’s special issue on RRI. The outcomes of capacity-increasing projects, such as PACITA, can help institutions increase their ability to practice RRI. Surveys (such as Eurobarometer on RRI, S&T) can be a useful source of information about public attitudes on science and technology. And it is always useful to know about possible negative consequences of failing to carry out RRI-compatible processes (see Late Lessons from Early Warnings). Institutions can also adopt indicators to gauge how they are doing in implementing RRI (e.g., Indicators for Promoting and Monitoring RRI) and make use of specific evaluation tools (e.g., NCCPE EDGE tool for institutional reflection on public engagement).

RRI’s key agendas

RRI embraces six policy agendas: Governance, Ethics, Gender Equality, Open Access, Public Engagement and Science Education. All six come into play to a greater or lesser extent depending on the challenges being addressed. You can evaluate where you and your team are regarding each agenda by using the Self-Reflection Tool.

The following resources can help you advance the agendas:

• normative frameworks that include RRI principles, such as EPSRC’s Framework for Responsible Innovation;

• codes of conduct, such as those provided by the US Office of Research Integrity, and training tools aimed at identifying and avoiding fraud and misconduct, such as Training and Resources in Research Ethics Evaluation (TREE);

• plans that support structural change for gender equality in both decision-making bodies and labour conditions (e.g., the European Commission’s Vademecum on Gender Equality in Horizon 2020, Yellow Window’s Gender in EU-funded Research toolkit, the EC’s report on elements and solutions for structural change in research institutions and the INTEGER guidelines for structural change in higher education and research organizations);

• measures to promote open access (e.g., ALLEA’s statement on the enhancement of open access, the UNESCO guidelines) and facilitate researchers’ use of open access repositories (e.g., Registry of Open Access Repositories (ROAR)), as well as internal policies to promote transparency and openness across the R&I process (e.g., TOP Guidelines);

• plans for fostering dialogue, reflection, and participation within institutions (e.g., Sciencewise’s Departmental Dialogue Index, Involve’s People & Participation tool) and for supporting structural change towards public engagement (e.g., NCCPE’s Planning for Change, the Concordat for Engaging the Public with Research); resources on using participatory methods to widen dialogue (e.g., Engage2020’s Action Catalogue, the King Baudouin Foundation’s Participatory Methods Toolkit, the UK government’s Open Policy Making toolkit, Sciencewise’s The Use of Experts in Public Dialogue) and on evaluating those methods (e.g., the RCUK’s Guide for Evaluating Public Engagement Activities);

• understandings that all citizens are involved in RRI, to a greater or lesser extent, as discussed in Societal Issues in Social Studies and Science Education: Promoting Responsible Citizenship, Sciencewise’s Enabling and Sustaining Citizen Involvement, and the OECD’s Focus on Citizens.

Walk the walk: Fostering RRI

At the European level, RRI is a key element of the European Research Area and the Horizon 2020 programme, both of which are strongly geared towards wealth creation and economic prosperity. RRI addresses the EU’s Grand Challenges to bring the benefits of R&I to its citizens and to involve them in ensuring the system addresses their needs, concerns and aspirations. These considerations also apply to the national level. Thus, ministerial teams with responsibilities in all fields of science and research, business and innovation, and education and training can benefit from incorporating RRI and adapting it to their needs.

Future-oriented governance is fundamental to safeguarding the creation of effective and innovative solutions to social issues. What are the key features of anticipatory governance? Anticipatory governance encompasses the elements –foresight, engagement and integration– for which you can find information, inspiration and practical tools in the RRI Toolkit.

Foresight and assessment of technologies in the process of emerging

The possible impact of emerging technologies and scientific developments has been the subject of study for decades in (parliamentary) technology assessment, a scientific, interactive and communicative process that aims to contribute to the formation of public and political opinion on societal aspects of science and technology. Despite the many differences between the existing organisations, they are united by the use of methodologies to execute technological foresight and forecasting. For examples, check out Engage2020’s Action Catalogue selection of future-oriented governance tools, such as future panels, future search workshops, and world/science cafés. Other methods for future-impact assessment are scenario workshops, collaborative inquiry and fictive scripting. Finally, the UK government’s Foresight Horizon Scanning Centre has published a toolkit of 19 techniques and methods for strategically identifying and analysing future risks and opportunities.

Traditional risk assessment and forecasting will not be sufficient for obtaining an integral assessment of all ethical, legal, and societal aspects (ELSA), as well as the opportunities and pitfalls. This message, powerfully added to the agenda by think tanks like DEMOS, endorses the importance of development and research into the ELSA of emerging sciences and technologies that is taken further by the framework of RRI.

The European Commission is keenly interested in foresight and developing strategies for key enabling technologies and emerging technologies. This has resulted in the EC’s support of mainstream RRI and its formulation and financing of many projects that foster RRI.

Engagement of publics that are mostly latent

The RRI Toolkit encompasses many deliberative tools to reveal and explore uncertainties, motivations, visions, questions, resistances and dilemmas. It facilitates ‘upstream’ public engagement via dialogues that integrate societal needs, ideas and expectations into policy debates. The number and types of engagement methods are multiple and growing. They can focus on one well-defined issue and completed within a limited timeframe like focus groups. Or they can stretch over a longer period, allowing for more detail through, for example, public consultations, consensus conferences, citizens panels, value labs or even town meetings. Less structured methods are the debates that take place during science festivals, open spaces and world cafés. More information on engagement can be found in this Toolkit under Fostering Multi-stakeholder Engagement.

The implicit assumption of all these methods is that public engagement should no longer be seen as a ‘brake on progress’ but as a way of maintaining and renewing the social contract that supports science. However, there is rising concern about the perceived tendency for leaders to keep control over the outcomes of public engagement or to ritualise the debate.

Integration of broader considerations into research and development (R&D) contexts

Upstream engagement is logically followed by ‘midstream’ engagement: innovation workshops focused on the R&D practices at the heart of the research and innovation (R&I) enterprise. This valuable embedding of reflection –with organisations devoting time, energy and resources– can take place through, for example, value sensitive design, participatory design, reflexive interactive design, design thinking, citizen labs and living labs. In these projects researchers, projects and subfields are linked together and to the context “they seek to study with the aim of incrementally building the capacity to more broadly anticipate and participate in shaping things to come” (Barben et al., 2007, Handbook of Science and Technology Studies).

Future-oriented governance and decision making

When dealing with emerging technologies, there is always the dilemma of long-term planning requirements and short-range acceptance problems that surround emerging and uncertain developments in science and innovation. The Collingridge Dilemma, as this problem in known, explains that, in the early stages, new technologies can be adapted relatively easily, but their social consequences are still uncertain. As a technology matures its potential adverse uses and consequences manifest; however, societal control over the technology is now limited, time consuming and expensive. How can future-oriented governance help ensure that stakeholders act responsibly but also drive performance?

It is crucial to synchronise with existing decision-making cycles: being ‘in sync’ and tuned to the right moment in the circle of governance (stage gating) can result in clear and sound policy. Therefore, it is important to link the anticipatory governance tools mentioned above to the existing planning cycle while also making it flexible enough to adapt to existing situations and needs (i.e., midstream modulation, such as in the NanoDiode project).

To do this, the timely and conscious involvement of stakeholders is essential. All actors should perceive the sense- and decision-making processes as legitimate, transparent and trustworthy. Interesting scenarios and lessons are offered in the Responsibility Navigator, developed by the Res-AGorA project. It is designed to support the identification, development and implementation of measures and procedures that can transform R&I in such a way that responsibility becomes an institutionalised ambition. Another very useful online support tool that can help you select the optimal project methodology for future-oriented policy analysis is offered at Doing Foresight.

Integrating the values, guiding principles and requirements of RRI in a funding scheme, whether existing or brand new, can contribute to increasing the ethical acceptability, sustainability and social desirability of the processes and outcomes of a programme. This section offers guidance for developing a funding policy aligned with the RRI principles.

Before the project: Defining the scope of the call

1. Setting priorities with stakeholders

   Sometimes public consultations and multi-stakeholder dialogues are organised by the funders of R&I themselves prior to setting research goals in specific funding calls. A participatory and deliberative process with concerned stakeholders can assist policymakers and researchers in effectively targeting R&I to (unmet) societal needs, setting R&I priorities and including the potential users of R&I results. Different methods are available, such as those in the following examples:

   • Priority Setting Partnerships in health research (Dialogue Model), James Lind Alliance (UK)

   • Multi-stakeholder dialogue for priority setting in health research (Mind the Gap!), King Baudouin Foundation (Belgium)

   • Engaging Citizens to Shape EU Research Policy on Urban Waste, Voices project

   • How to Set Up Collaboratively a Research Agenda, in this Toolkit, provides more information

2. Funding R&I that integrates the RRI principles

   There are different ways in which RRI can be embedded in a funding call. The first step is supporting active and balanced promotion of science to the public, as is done by the Luxembourg National Research Fund or the Swiss National Science Foundation. The next step may be to consider funding stakeholder involvement (e.g., UK’s Bioscience for the Future). The most ambitious approaches so far include RRI criteria or standards in the call (e.g., Dutch Responsible Innovation programme) or focus on practices -projects and programmes- in which the RRI process dimensions are successfully integrated throughout the R&I process (e.g., EFARRI, the European Foundations Awards for RRI).

During the project: Fostering RRI through criteria and guidelines

​

1. Embracing the RRI holistic approach

   The ProGReSS project’s RRI Funder Requirements Matrix offers an overview on how some R&I outcomes (ethical acceptability, sustainability and societal desirability) can relate to different instruments and appear at different stages of the R&I process depending on the funding strategies analysed (those in Australia, China, EU, India, South Africa and US). For example, on the first level, RRI outcomes may operate as compliance rules that those who receive funding must adhere to, such as:

   • a prerequisite for R&I in order to obtain funding (e.g., obtaining ethics committee approval, applying gender equality plans, accepting fair animal research practices, adopting open data management plans or ensuring societal desirability)

   • a set of rules to be followed during the study (e.g., Wellcome Trust’s Policy and Position Statement)

   At the second level, public engagement becomes a priority for funders who implement R&I outcomes with the help of public engagement tools (e.g., Participation Compass). Funders might also request that researchers conduct public consultation programmes to ensure that the outcomes of any research or innovation will meet the expectations, values and needs of society. Finally, the third level comprises those programmes where R&I outcomes are themselves a research goal, such as the concept of sustainability across several of the countries analysed in the ProGReSS matrix.
   
   The European Foundations Award on RRI (EFARRI), launched in 2015, constitutes an up-to-date example of general selection criteria on RRI expertise, experience, process dimensions and organisational capacity. These criteria are based on RRI Tools’ quality criteria on RRI practices, but there are other criteria and indicators, such as those in the Quality Criteria and Indicators for Responsible Research and Innovation: Learning from Transdisciplinarity.


2. Incorporating the RRI policy agendas

   While incorporating RRI outomes into the funding strategy represents the most holistic approach, it is still not the most usual one. A more common and probably simpler way to embed RRI within a call is to include different requirements related to the RRI policy agendas. Funders can find a wealth of examples and resources to help them on this matter, especially on the three policy agendas that are more normative now:

   • Ethics and research integrity: The responsibility for ensuring that funds and resources are utilised optimally without any misconduct rests on the shoulders of the researchers, their institutions’ ethical committees, and the funding organisations. Thus, funding bodies refer to codes of conduct (e.g., Research Councils UK’s Policy and Guidelines on the Governance of Good Research Conduct, European Code of Conduct for Research Integrity, Universities UK’s Concordat to Support Research Integrity and the European Commission’s Ethics Screening, Review, and Follow-up and Audit). Another example is Training and Resources in Research Ethics Evaluation (TRREE), which provides basic training and capacity building on the ethics of health research.

   • Gender equality and diversity: The gender dimension is progressively being integrated into research content, as described in the Yellow Window’s Gender in Research Toolkit. In parallel, funders remain committed to attracting the best potential candidates from a diverse population, as shown for example by the Research Councils UK’s Action Plan for Equality, Diversity and Inclusion and the Engineering and Physical Sciences Research Council’s (EPSRC) grant terms regarding equality and diversity.

   • Open Access: A growing number of funding schemes incorporate requirements to enhance the openness and transparency of R&I processes. Useful resources can be found at the OpenAire project’s section for funding agencies, the Pasteur4OA Toolkit with case studies for research funders, and the MedOANET project’s Guidelines for Implementing Open Access Policies for Research Funding Organizations.

3. Assisting applicants through guidelinesp

   Funders usually provide guidelines or a framework to help researchers and innovators address the R&I practice criteria of the call when applying for funding. In the case of RRI principles, this may include such items as openness, communication, ethical acceptability, research integrity and gender equality. Examples of these guidelines are the Wellcome Trust’s Guidelines on Good Research Practices, the UK Medical Research Council’s Guiding Ethical Research Practice and the (Danish) Lundbeck Foundation’s Code of Good Research Practices. The UK’s EPSRC has adopted a Framework for Responsible Innovation that involves the “AREA process” (Anticipate, Reflect, Engage and Act) to help researchers consider societal issues that may be involved with or flow from their work. To develop such guidelines for RRI, funders may also find useful the RRI Tools’ quality criteria on RRI practices and the Self-Reflection Tool.

After the project: Assessing and evaluating

Different funding schemes use different methods to assess and evaluate proposals. Depending on the method chosen, the evaluation team may need to consist of people with different types of expertise:

• Jury: The jury should be composed of experts familiar with the RRI concept or similar concepts and with the specific field of the call/proposal. Institutions may thus need to build a pool of RRI experts, sorted by discipline, which could be contacted for evaluations. The gender dimension should be taken into account in the jury’s composition to minimise gender bias. The jury can be involved before research (selection) and/or after research.

• Stakeholders/End users: Stakeholders and end users can be involved in the assessment or the evaluation. However, to avoid tokenism it is crucial to clearly define their role in the assessment. For example, the Hart Foundation in the Netherlands has a Commission for Societal Quality that assesses the societal impact of research.

• Peer review: Core principles of peer review are excellence, impartiality, transparency, efficiency, confidentiality, appropriateness for purpose, and ethical and integrity considerations, as described in the European Science Foundation’s European Peer Review Guide or the UK Medical Research Council’s Peer Review Policy. The inclusivity and engagement of stakeholders could be added to these principles. Therefore, criteria are needed to evaluate public engagement (the plan; use of results; different publics, including less heard voices; etc.), such as those proposed in the article Which Indicators for the New Public Engagement Activities?, in the reports Indicators for Promoting and Monitoring RRI and Metrics and Indicators for RRI, or in the RRI Tools’ quality criteria on RRI practices.

One step further: Research uptake and policy advice

Research uptake includes all the activities that facilitate and contribute to the use of research evidence by policy makers, practitioners and other development actors. The UK government’s Research Uptake guide offers practical advice on how to build a functional communications interface between researchers and policy makers. The European Commission has also published a guide on Communicating Research for Evidence-Based Policy Making.

Why PE?

Besides the reasons explained in the What is RRI? section of this Toolkit, there are different purposes for engaging the public in R&I. Some of these are summarized in the Engage2020 project’s eAnthology:

• gauging public opinion on a particular science project/issue or a new technology;

• assessing a new technological application;

• helping researchers gather data for a given project;

• having a representative sample of people make judgments or decisions that might inform policy making;

• getting the public and experts to co-create knowledge or co-produce innovation.

Who should be engaged?

Researchers, research institutions and public authorities have traditionally led PE activities. However, the third sector, or social sector, has been increasingly involved at different levels of R&I and policy making, giving access to their interests, viewpoints and experiential knowledge. The current trend is to also engage the fourth sector, an emerging sector composed of actors or groups of societal actors that cooperate through hybrid networking. Depending on whether PE focuses on the third or fourth sector, it is often labelled as stakeholder engagement or citizen engagement, respectively. In either case, PE activities are evolving from linear and bilateral collaborations towards dynamic, networked, multi-collaborative innovation ecosystems. Engage2020’s eAnthology offers different methods for selecting public participants according to the purposes of the engagement process.

When to conduct PE

Depending on the goals, PE can be applied at multiple phases of the R&I process. However, to guarantee that engagement truly influences R&I, PE should be conducted iteratively, with feedback loops. Engage2020 provides brief descriptions of PE at different places in the R&I process:

Before starting the R&I process

Program definition: Setting the R&I agenda

How to Set Up a Participatory Research Agenda describes experiences where collaborative deliberation techniques have helped different stakeholders integrate and prioritize R&I topics to collaboratively set the agenda of R&I programmes.

During the R&I process

Project definition: Defining the R&I process with permanent adjustments

Engagement activities should be designed to give citizens the opportunity to contribute their specific knowledge through deliberative processes. Citizen panels, such as SUGAR or CAC, promote this sort of engagement. Stakeholders can contribute their practical knowledge to improve technologies or products through methods such as open innovation and structures such as living labs.

Project execution: Co-developing R&I

In this phase, the R&I itself is conducted through a collaborative approach that equitably involves diverse partners. This sort of process often starts with a problem important to a community, civil service organization or other group (e.g., minorities, associations). The problem is then developed and framed as a research question by combining/constructing knowledge from the scientists or innovators and the different partners. Examples of engagement processes within this phase include community based research and citizen science projects where the involvement is not restricted to data collection.

After implementing the R&I process

Supporting participatory policy development

There is a growing trend in science and technology policy to accompany decisions with democratic practices and broad precondition and foresight analyses. These practices and analyses are aimed at gauging the risks, benefits, and ethical, legal, environmental and socio-economic impacts of new technologies. Different communities in Europe are working in this direction (see the PACITA project for an example of technology assessment).

How to conduct PE

PE2020's report Innovative Public Engagement describes five main categories of public engagement (listed in the table below), three of which involve one-way communication. In its eAnthology, Engage2020 states that one-way communication processes should not be considered public engagement. To them, public engagement is about two-way communication. The literature also describes different engagement categories based on the level of participation. Among others, it differentiates between:

• consultation (obtaining public feedback);

• involvement (working with the public and considering its input);

• collaboration (partnering with the public in each aspect of decision making);

• delegated power (letting the public decide).

*See Action Catalogue, Participedia, Participation Works or PACITA to find the best method for a specific project.