School feeding, moving from practice to policy: reflections on building sustainable monitoring and evaluation systems

Abstract

Objective

To provide an overview of the status of monitoring and evaluation (M&E) of school feeding across sub-Saharan Africa and to reflect on the experience on strengthening M&E systems to influence policy making in low-income countries.

Design

Literature review on the M&E of school feeding programmes as well as data from World Food Programme surveys.

Setting

Sub-Saharan Africa.

Subjects

Countries implementing school feeding.

Results

Only two randomized controlled impact evaluations have been implemented in sub-Saharan Africa. Where M&E data collection is underway, the focus is on process and service delivery and not on child outcomes. M&E systems generally operate under the Ministry of Education, with other Ministries represented within technical steering groups supporting implementation. There is no internationally accepted standardized framework for the M&E of school feeding. There have been examples where evidence of programme performance has influenced policy: considering the popularity of school feeding these cases though are anecdotal, highlighting the opportunity for systemic changes.

Conclusions

There is strong buy-in on school feeding from governments in sub-Saharan Africa. In response to this demand, development partners have been harmonizing their support to strengthen national programmes, with a focus on M&E. However, policy processes are complex and can be influenced by a number of factors. A comprehensive but simple approach is needed where the first step is to ensure a valid mandate to intervene, legitimizing the interaction with key stakeholders, involving them in the problem definition and problem solving. This process has been facilitated through the provision of technical assistance and exposure to successful experiences through South–South cooperation and knowledge exchange.

School feeding programmes are ubiquitous in high-, middle- and low-income countries. A joint analysis developed by the World Bank, the World Food Programme (WFP) and the Partnership for Child Development (PCD) identified that every country is in some way and at some scale seeking to provide food to its schoolchildren⁽¹⁾. It is estimated that over 300 million primary-school children are covered and the social investments in the programmes are substantial. Across the globe, a key response to the recent economic crisis has included the scaling up of school feeding as a safety net for children living in poverty and food insecurity. School feeding is, however, a complex intervention and designing effective programmes requires careful management of trade-offs among design objectives, targeting approaches, feeding modalities and costs⁽²⁾.

Bringing national school feeding programmes up to scale requires considerable resources and a steady flow of funds: in low-income countries, school feeding programmes on average cost about $US 50 per child per annum⁽³⁾. In this context, strengthening monitoring and evaluation (M&E) systems is of paramount importance to maximize programme impact and minimize costs. Unfortunately, in practice these systems are generally not well developed. Strong political will and technical expertise are needed to support low-income countries where national governments scaling up school feeding programmes face budget constraints that often limit investments in M&E.

The present paper aims to provide an overview of the current status of M&E of school feeding across sub-Saharan Africa and reflects on the recent experience on strengthening M&E systems to influence policy making in low-income countries.

Methods

We undertook a review of the recent academic and grey literature on the M&E of school feeding programmes. Databases were searched to identify relevant papers as well as data from school feeding surveys from WFP that were undertaken during the period 2002–2012. The review focused mainly on capturing the typology and resources invested in the M&E activities.

Findings

Recent experience indicates that during the scaling up of school feeding programmes in low- and middle-income countries, investments in M&E are often overlooked. In sub-Saharan Africa for example, over the last 5–10 years, only two randomized controlled trials – the gold standard for impact evaluations – have been implemented (in Burkina Faso and Uganda) and neither of these covered government programmes. A number of quasi-experimental evaluations have been undertaken across the region, mostly by WFP; however, these studies are limited by the systematic lack of baseline data collection and difficulties in setting up adequate control groups⁽⁴⁾.

An ongoing review of M&E systems for school feeding in the ten countries with national school feeding programmes in the continent shows that where M&E systems for school feeding exist, they generally operate under the Ministry of Education, with other Ministries (e.g. Health, Agriculture, Local Development) usually part of a cross-ministerial technical steering group supporting programme implementation.

In countries where M&E data collection is underway, the focus is mostly on process data and service delivery and not on child education or health outcomes (Fig. 1). As a result, cost data have recently become more readily available, but there is a dearth of evidence on cost-effectiveness of programmes⁽⁵⁾. There is generally no dedicated budget line at national level: M&E is usually part of the administration, support or overhead costs for the programme. Moreover, in most countries examined, the M&E system is not underpinned by a national policy, plan or project document.

Fig. 1 (colour online) A focus on monitoring process? Snapshot of monitoring and evaluation of school feeding programmes in sub-Saharan Africa

In terms of indicators, there is to date no internationally accepted standardized framework for the M&E of school feeding programmes, reflecting the varied and complex set-up of school feeding programmes worldwide. At regional and country levels though, the Institute of Statistics of the United Nations Educational, Scientific and Cultural Organization has recently been making efforts aimed at harmonizing monitoring systems across the education sector, including for school feeding, also strengthening the Ministry of Education's ‘Education Management Information Systems’ (EMIS). To date, however, there have been only few examples, including Tanzania, of functioning M&E systems for school feeding that are integrated into EMIS.

This preliminary snapshot of school feeding M&E points to a number of lost opportunities to improve programme responsiveness, efficiency and effectiveness. The lack of investment in M&E has also provided a constraint in terms of learning from the considerable experiences in-country and across the region, both in the programme and in the policy sphere. There have been some examples where evidence on programme performance has influenced policy and programme, most notably in terms of cost-efficiency where data are more readily available. For example, the publishing of cross-country comparisons and benchmarks of school feeding programmes using 2005 data resulted in the country at the most costly end of the spectrum (in this case Benin) to redesign its food basket and reduce costs by a factor of four when the benchmarking exercise was repeated in 2008⁽⁶⁾. Considering the popularity and the size of the investment in school feeding though, these cases are more anecdotal than the norm, highlighting the opportunity for more systemic changes in the policy support processes.

Discussion

Influencing policy processes

Can the recent rethinking of school feeding programmes across the world provide an opportunity to strengthen evidence-based policy making and build sustainable M&E systems? Emerging trends suggest that programmes in low- and middle-income countries are in a transition to sustainability and national ownership⁽¹⁾. From the policy perspective, governments in sub-Saharan Africa have included school feeding programmes linked to agriculture development within the African Union Comprehensive Africa Agriculture Development Programme⁽⁷⁾. School feeding programmes are being financed with investments of the order of millions of dollars per annum by national governments in at least ten countries on the continent. In response to this opportunity and policy-level engagement, the PCD, World Bank, WFP and other stakeholders have been working together to strengthen national school feeding programmes. The partnership has been undertaking analyses in a number of countries designed to develop a better understanding of the school feeding system in its different context-specific configurations⁽⁸⁾. One of the key themes in the dialogue on strengthening M&E with national governments was to highlight the opportunity to understand the cost-efficiency and effectiveness of school feeding, with a view to increasing the impact and value for money of the investment in the programme.

Policy processes for school feeding are complex and can be influenced by a number of factors. The present discussion reflects over a decade of experience with building M&E systems for policy and programme support for school feeding by WFP and PCD in over thirty countries in sub-Saharan Africa. Figure 2 describes four elements for influencing policy that have emerged from this experience. Awareness of the particular policy processes in each country context is paramount: building the knowledge base on school feeding is only one piece of the policy-uptake puzzle, which involves interactions between a wide range of stakeholders across different sectors, appropriate use of the context (i.e. elections, incoming ministers, political commitments and economic opportunities) and inclusion of school feeding in relevant policy discussions (Fig. 2).

Fig. 2 (colour online) Elements of the policy-making process

Adapting the work on evidence-informed policy making in the health sector⁽⁹⁾, we captured some of the key interactions between actors, context and content involved in the policy-making process and identified three important entry points.

1. 1. Obtaining a mandate from government to address the issue of weak M&E: a first step to ensure legitimacy and long-term sustainability of the process and an opportunity to begin engaging with policy makers.

2. 2. Providing technical support: context-driven (e.g. short-term capacity gap filling and medium-term capacity development activities based on assessment of gaps in the current system).

3. 3. Facilitating inclusive South–South partnerships: politicians listen well to what other politicians are doing. The school feeding experience in Latin America and South Asia provides a wealth of opportunities for innovations that are more likely applicable to the low-income context than experiences in the North. In addition, such South–South partnerships have less ‘political baggage’ in terms of the recent history in the continent and may act as catalysts for more conducive environments for policy uptake.

Ensuring a valid mandate before initiating action, validating the value added of the initiative to strengthen M&E systems by providing meaningful technical assistance and promoting awareness on successful experiences through South–South cooperation have been successful ways to influence decision making in the supported countries (Fig. 3). This process is illustrated in a very recent example provided below.

Fig. 3 (colour online) Entry points in the policy-making process⁽⁹⁾

Building sustainable monitoring and evaluation systems

In Mali, the national programme is currently covering 651 primary schools located in food-insecure areas, with over $US 5 million committed per annum from the national budget. One of the focus areas of the PCD–WFP–World Bank partnership activities has been to strengthen the M&E of the national programme as it scales up. As a first step in the process, a Memorandum of Understanding was signed in late 2010 with policy makers in the Ministry of Education that outlined the mandate and the scope of the work. This paved the way for stakeholder engagement activities during 2011, working across the traditional disciplines of agriculture, education, health and nutrition, including policy makers, practitioners, researchers and civil society. The country-level activities fed into the development of an integrated assessment that followed a standard approach to examine school feeding programmes across five quality standards, namely: design and implementation; policy frameworks; institutional capacity and coordination; funding; and community participation. The scoping analysis also provided the basis to understand the key gaps in the evidence base and to shape a structured research agenda.

As a result, a national school feeding M&E plan is currently being developed in Mali under the leadership of the national school feeding centre in the Ministry of Education⁽¹⁰⁾. The national M&E plan will describe the national M&E strategy to be implemented over the national school feeding programme cycle, detailing the different components of the M&E system in place or planned for to monitor and evaluate the programme performance. A key element of the M&E plan is a randomized controlled trial that was designed around the scale-up of the national programmes, with baseline procedures underway in January 2012⁽¹¹⁾.

The experience in Mali suggests that policy making was influenced and a concrete plan for change was obtained through a participatory process. Stakeholders across sectors are now engaged in the systems strengthening activities covering broader elements of programme design, management and implementation, adding to the sustainability of the plan and supporting the transition to national ownership. Similar activities are being undertaken in Ghana and Kenya, with impact evaluations planned for late 2012. The challenge is now to maintain the momentum on implementation, ensuring that the value added to the process materializes through the provision of meaningful feedback into the programme, improving impact while reducing losses and costs. These elements remain important areas for future research.