Strategic Foresight and Policy Evaluation: Insights for an Integrated Approach

Abstract

Grand challenges are shaping twenty-first-century politics. Threats connected to health, climate, demographics and welfare are increasingly intruding on the lives of citizens. Still, governments are often found off-guard, and policymakers need strategies grounded in longer-term perspectives. Strategic foresight (SF) helps us to design and shape policies to prepare to withstand shocks, anticipating and adapting to changes. However, as governments work towards embedding SF into their policymaking processes, the empirical evidence suggests that applications are still piecemeal and predominantly limited to the agenda-settings and policy formulation stages. In this article, we argue that to drive anticipatory governance, foresight needs to be applied at all stages of the policy cycle, including in evaluating policies to draw lessons for future interventions. We maintain that considering SF systemically throughout the policymaking cycle, from agenda setting to evaluation, strengthens anticipatory governance.

I. Introduction

In its communication on Better Regulation, the European Commission recognised that strategic foresight (SF) is crucial in “future proofing” European Union (EU) policymaking. ¹ SF ² can be defined as a set of methods that support decision-makers in analysing possible, plausible and preferable futures. ³ The aim is to provide Member States and EU policymakers with tools that help anticipate future trends, thus increasing their adaptiveness and ability to respond to crises. In this regard, its contribution to strengthening anticipatory governance is well documented and understood. SF has been widely used to identify critical issues and risks that are likely to affect a country or territory, or it can be used to create participatory visions for desirable collective futures. Furthermore, proponents of SF have advocated for its direct applicability to informing policymaking.

In principle, as it creates future-regarding mind-sets and encourages critical thinking, SF can enrich all policy cycle stages, including impact assessments, fitness checks and policy evaluations. However, the empirical evidence suggests that achieving full integration of SF insights and outputs within the policy cycle is challenging. ⁴ SF tends to result in piecemeal or one-off exercises, often conducted at the early stages of the policy cycle (eg agenda setting or policy formulation). This curtails SF’s potential to underpin policymaking, foster learning and support anticipatory governance. ⁵ In particular, a lack of follow-up and continuation of SF during the later stages of the policy cycle, namely policy evaluation, increases the risk of SF being ineffective in driving transformative anticipatory governance. Indeed, even academic studies predominantly focus on SF as an input to the early formulation of policies and strategies. As a result, its contribution and applicability to policy evaluation are less clearly understood.

This article aims to unpack the relationship between policy evaluation and SF. We highlight that a stronger integration between these often-neglected practices could foster better policymaking and anticipatory governance. We look at the cases of Wales, Portugal and the EU, which are at the forefront of the adoption of SF and have set up three different approaches to integrating SF into policymaking and anticipatory governance. Looking at three organisational structures helps us to pinpoint the potential for further integration between SF and policy evaluation. Cross-cutting themes emerge by looking at policy evaluation through a SF lens (and vice versa), including the opportunity to foster learning and generating collective knowledge in areas of public interest through participatory practices.

We maintain that SF as a participatory, prospective and policy-orientated practice ⁶ should continue to be implemented in policymaking, but more emphasis must be placed on the evaluation phase and inclusive and deliberative citizen involvement.

II. Policy evaluation and foresight: closing the policy cycle

The case for integrating SF into the evaluation phase is straightforward. Policy evaluation is intended to have two main objectives: one is accountability, meaning to hold to account policymakers and bureaucrats; and the other is learning, which applies more to stakeholder learning from evaluation findings. ⁷ The learning purpose of evaluation is to encourage the “uptake of findings” ⁸ ; in this sense, a minimal definition of learning is the “update of policy beliefs” ⁹ that results from some evidence-based analysis.

As an instrument of anticipatory governance, future thinking in evaluation would specifically trigger reflexive learning. This kind of learning happens if there is a certain predisposition to listen and, most importantly, to reconsider one’s preferences. ¹⁰ This is what happens when decision-makers consider evidence coming not only from retrospective evaluation methodologies but also from prospective methods (ie foresight).

Ribeiro and Weiner suggested integrating the past while considering the future impacts of a policy. According to these authors, policy evaluation should inform policymakers about the different impacts and logics of change in the policies and how these policies could adapt to changing contexts. This “learning” should then inform foresight analysis, which should then feed into impact assessment. ¹¹ This is defined as “foresight from hindsight”. ¹² In applying foresight, decision-makers and stakeholders should learn from past experiences to sketch future scenarios and action plans. In policymaking, this means that ex post evaluations should not only be informed by future trends but also inform foresight exercises.

Lastly, when embedding SF in policymaking and policy evaluation, it is important to consider inclusive and appropriate citizens’ and stakeholders’ engagement to ensure that a broad range of views and voices are represented and can actively contribute to a foresight-informed evaluation and learning process. As the Organisation for Economic Co-operation and Development (OECD) stresses, this enhances the legitimacy of policy interventions and outputs. ¹³

While there is broad consensus on the benefits of opening up the policymaking process to external inputs, the main issue regards the modalities and the recurring impediments to stakeholders’ mapping, participation and feedback. Especially regarding what concerns foresight and policy evaluation, according to the Indicators of Regulatory Policy and Governance (iREG), ¹⁴ we know that few countries consult stakeholders whilst reviewing and evaluating existing regulations; in this sense, real embeddedness of foresight into policymaking should be accompanied by overall improvements in conducting ex post evaluations, and this should be integrated into the policy cycle.

Well-designed citizen and stakeholder consultation processes increase the overall quality of policies and enhance trust in governments as forms of public administration. In a recent study ¹⁵ by the Committee for Scientific and Technological Policy of the OECD, the authors investigated why and how citizens engaging in innovation could help define future and long-term policy priorities. It is recognised that citizens’ and stakeholders’ engagement plays an extremely important role in enhancing trust in and adaptation of policies; however, it is also crucial that foresight methods are used while engaging citizens and stakeholders. This means exploring multiple scenarios and their implications, developing a shared vision for the future and co-developing action plans.

III. How to embed foresight into policy evaluation

Combining SF with evaluation methodologies requires establishing a future-regarding mind-set. ¹⁶ Few examples offer blueprints for combining foresight and evaluation. Among these, the work by Sitra ¹⁷ looks at how foresight can be used to shape evaluation practices and directly improve policy outcomes. As a minimum, foresight can be deployed to back-cast actions, milestones and indicators for evaluations starting from multiple exploratory or archetype scenarios that consider best- or worst-case policy developments. Horizon scanning or policy interventions can be evaluated against long-term trends, weak signals and wildcard/black swan events through future-facing stress-testing mechanisms. Foresight can also shape the way evaluation is framed in the first place, and in so doing strengthen its ability to handle the long-term nature of the problems that the policy set out to tackle. ¹⁸ In a report published by the UK Government Office for Science (GOScience), there are examples of what future-focused evaluation questions could look like: “were the assumptions underpinning our policy robust?” and “were there any unforeseen events, societal responses or disruptors our policy could not adapt to?”. In addition to embedding long-term perspectives, foresight methods can also be used to deal with complexity in policy evaluation. The Magenta Book of 2020 ¹⁹ on policy evaluation lists methods related to foresight as tools that can help us to incorporate the time dimension. For example, computational system modelling can be combined with scenario planning to understand how sustainable a policy change is expected to be in the longer term, and the use of scenarios is presented as a way to communicate uncertainty, risks and opportunities for more accurate communication and more effective learning. In line with this, Bana e Costa et al ²⁰ developed a framework incorporating scenarios for modelling uncertainty for policy evaluation through foresight. This is one way to establish SF thinking, avoiding cosmetic implementations of methodologies. Therefore, when thinking about anticipatory governance, it is only through systemic integration of SF and future thinking into the ex post evaluation phase that policymakers, stakeholders and desk officers can acquire knowledge from retrospective reviews while designing future-orientated activities.

IV. Embedding strategic foresight: the cases of Wales, Portugal and the European Unions

We now look at the cases of Wales, Portugal and the EU, regions that have set up three different approaches to integrating SF into policymaking. We will look at three different organisational structures, with a focus on the evaluation phase. This will help us pinpoint some common features but also ways forward. Institutionally speaking, in Wales, the Well-being of Future Generations (Wales) 2015 Act (henceforth, “the WFGA”) sets explicit commitments to socio-economic, cultural and environmental well-being. It identifies long-term thinking as one of five key ways of working that Welsh public-sector organisations bound by the WFGA need to show that they are applying to pursue the well-being objectives.

In recent years, the Portuguese government set up the Centre for Planning, Policy and Foresight Competence of Public Administration (PlanAPP), tasked with driving applications of SF throughout the policy cycle, from planning to evaluation. The European Commission has charged its vice-president to lead the efforts to embed SF into policymaking, introduced SF as one of the tools for better regulation, mandated the Regulatory Scrutiny Board (RSB) to check on SF and outlined methodological steps that policymakers can take to integrate long-term perspectives into policy.

1. Wales

Wales is a devolved legislature in the UK. Since 2015, the Welsh policy landscape has been shaped by the WFGA. The WFGA sets in law seven national well-being goals underpinned by the principle of sustainable development comprising five ways of working sustainably. The WFGA aims to improve the social, economic, environmental and cultural well-being of Wales to give current and future generations a good quality of life. The WFGA currently applies to forty-eight public bodies across Wales, including the Welsh Government. The duty requires public bodies to set well-being objectives and maximise their contributions to them. They have to think about the long term, work better with people and communities and each other, look to prevent problems and take a more joined-up approach. For example, public bodies need to show how they apply the five ways of working and how proposals fit with the well-being objectives in their integrated impact assessments (IIA). An IIA includes a suite of impact assessments that are used either to meet legislative obligations or to support policy interventions. The aim is to enable an integrated and rounded view of the impacts of any policy proposal. An IIA is most effective when it commences early in the policy development stage, having the ability to influence decisions and actions that form part of the policymaking cycle and ultimately leading to better impacts. Effective IIA throughout the policy cycle can shape how proposals’ delivery and success are defined and monitored and provide a basis for future evaluation. A long-term perspective needs to be considered while conducting IIAs, and therefore there is an opportunity to use this as a mechanism for framing policy evaluations through a SF lens.

Existing resources can support public bodies in integrating SF into their policymaking processes. For example, the Future Trends Report Wales, published by the Welsh Government following the Senedd elections, analyses key trends that may affect the future well-being of Wales. The evidence presented and signposted within the report can be used to stress test policy proposals as well as to inform policy evaluation and learning within IIAs or as a practice more generally.

2. Portugal

The use of SF in the Portuguese Government was already present during the twentieth century, with a focus on coordinating high-profile initiatives and government strategies driving long-term thinking. A dedicated government department (the Department of Planning and Foresight) used to be in charge of these functions, but it was closed in 2012.

More recently, in March 2021, the Portuguese Government recast foresight functions in government and created PlanAPP, which sits under the Ministry of Presidency. According to Legislative Decree 21/2021, ²¹ PlanAPP’s overarching goal is to strengthen the policy process, including through better applications of foresight in the policy cycle, from planning, to implementation and evaluation, and across government departments. In line with the historical tradition in this country, the approach taken by PlanAPP is characterised by a focus on coordinating and integrating policy and planning activities. Through its direct contribution to key strategic documents, such as the Major Options (Grandes Opções), and its remit, which also covers implementation and policy evaluation, PlanAPP is well placed to integrate foresight into all stages of the policy cycle. The role of PlanAPP as a catalyst for integrated policymaking and foresight is enhanced by its efforts at enhancing cross-departmental knowledge exchange and collaboration both with international organisations (eg the EU or the OECD) and with other countries and through the work of the inter-ministerial network (RePLAN). While work is still in progress, the organisational settings in which PlanAPP operates have the potential to explore applications of foresight to policy evaluation through a focus on integration, cross-departmental collaboration and learning.

3. The European Union

A significant change in the EU relates to the RSB, the independent oversight body checking on the quality of impact assessments and evaluations. In January 2020, with the new mandate, the RSB was set to examine the Commission’s work whilst considering the foresight dimension. Moreover, with the renewed Better Regulation toolbox, Tool #20 is dedicated to “Strategic Foresight for Impact Assessment and Evaluation”. The tool helps the services of the Commission to use foresight analysis systematically in ex ante and ex post evaluations in order to deal better with uncertainties. The EU support centre with competence over foresight methodologies is the Joint Research Centre (JRS). According to JRS, effective SF methods that policymakers can use are megatrends and future scenarios. Both of these methodologies are based on interactive workshops and desk research. According to Tool #20, megatrends should inform and shape the problem definition of the impact assessment analysis. Scenarios complement megatrends and can be used to assess how policies will perform in the future. The tools set out the guiding questions for identifying and considering megatrend findings in relevant sections of the impact assessment. Less information is devoted to fitness checks and evaluations, suggesting that evaluations should “look beyond the current relevance and reflect on how the key evolutions may affect the future relevance and coherence of the policy area”. ²² In the last RSB Report of 2022, the Board highlighted that SF elements were adequately considered in eighteen of the impact assessments that were scrutinised, and that foresight analysis helped to describe the relevant assessment in some evaluations. Moreover, the RSB has included a new indicator of SF in impact assessments and evaluations.

Simonelli and Iacob noted that the main issue regarding the inclusion of SF into the EU’s revised Better Regulation agenda was about the how rather than the what. ²³ In fact, with the Communication on Better Regulation, the Commission clearly stated that the foresight dimension should be salient during the whole policy cycle. Major trends and megatrends should thus be considered in the analysis of impact assessments, the consultation phase, fitness checks and evaluations. In this sense, stakeholders should have access to foresight tools and have more opportunities to provide feedback in the evaluation phase rather than only during the design phase.

As highlighted by Stephenson, ²⁴ however, in the EU, policy evaluation still needs to be genuinely open and inclusive to citizens. The EU scores higher than any other OECD country/region regarding consultation and evaluation. Especially given the new tools such as the online “Have Your Say” portal, citizens and stakeholders can access open consultations and provide their feedback easily; however, there is still room for improvement. One such area would be to integrate foresight systematically into consultations. Only by increasing future-regarding perspectives in the consultation phase and then feeding these into evaluations can SF be fully embedded into policymaking processes.

V. Conclusions

The cases of Wales, Portugal and the EU show that, in setting up SF systems, much effort is put into integrating long-term thinking throughout the policy cycle, but applications of SF during policy evaluations are still in their early stages. The three cases discussed here followed different approaches: Wales has integrated long-term thinking within the WFGA legal framework, whereas policy evaluation and learning rest primarily within the remit of IIAs. In Portugal, SF functions rest in PlanAPP, which aims to drive the application of SF to all policy cycle stages. Finally, the case of the EU shows how SF can be integrated into policy evaluation through the quality control carried out by the RSB. Indeed, all of these different approaches point to a need to further the analysis of the objectives of policy evaluation, but they also highlight a gap in our understanding regarding how the main actors and appropriate institutional settings can enable the integration of SF into policy evaluations. More empirical research is also needed to assess the actual learning mechanisms triggered and the inclusiveness of these processes.

Policymakers have an opportunity to strengthen their evaluation capacities through increased awareness of the contributions that SF could provide. Such integration has the potential to support anticipatory governance and enhance policy learning. SF can enrich the reach and scope of policy evaluation practices by broadening time horizons, challenging assumptions and considering policy effectiveness and outcomes through a future-regarding lens. As noted by Patton, foresight and evaluation can reinforce each other, as foresight scenarios require evaluative thinking and judgment; they share similar aims. ²⁵ In so doing, however, it will be crucial that SF remains participatory and inclusive throughout the policy cycle, including during policy evaluation.