This summary report was prepared by Penelope Jackson, under the guidance of Odille Keller and Samer Hachem Managers of OPEV 2 and under the overall leadership of Rakesh Nangia, Head of Operations Evaluation at the African Development Bank. The OPEV team was composed of Penelope Jackson and Samson Houetohossou.

The OPEV evaluation team was supported by a specialist team of consultants provided by the consortium Particip GmBHECDPM. The core consulting team was composed of Charlotte Vaillant, Dr. Apollinaire Ndorukwigira and Thomas Theisohn. The consultants’ team was supported by Tino Smail, Julia Schwartz, Fatten Aggad, Jean Bossuyt and Jan Vanheukelom. In-country support was provided by Isaac Ndungu (Botswana), Farrel Elliot (Sierra Leone) and Yvette Houngbo (Benin).

Objective

This evaluation was designed and conducted to focus on learning with the aim of helping the Bank to improve its support in this area. The specific objectives were to (i) draw evidence based conclusions on what the Bank is doing and achieving in this area (ii) identify lessons that can be used to improve support in the governance area(iii) learn from experience in the governance area for such work in other sectors.

Main Findings

• Over the decade project performance and results are mixed. Although over 80% of projects were rated as relevant at completion, figures were less robust for delivery of outputs and for timeliness. Less than half were rated as satisfactory for delivery of outcomes. The evaluation indicates that the gap between outputs and outcomes is related to Bank projects tending to support only some aspects of capacity and that objectives are not necessarily aligned to the timeframe and resources available.

• Projects spread across a large number of institutions or sub-sectors tended to perform less well than those with more focus. The case studies indicate that this relates both to the extra transaction costs involved in implementation and the effect of spreading limited funds too thinly; however, they also highlight the number of different institutions involved in supporting governance objectives.

• Results in fragile states – when it comes to delivery of outputs, Bank performance and borrower performance - are below average. However, the evaluation cannot confirm if this relates only to the more challenging context or also to the approach the Bank is taking.

• The timeframe has an influence. Projects which are initially designed with more realistic time frames tend to perform better than quick-fix attempts.

• The Bank has contributed to enhancements in institutional capacity and performance in institutions which are central to achieving good governance, though to varying degrees. Progress is most evident where the Bank has been engaged over a longer period and where support is comprehensive (whether from the Bank or in coordination with others’ support).

Main Lessons

• Understanding the context. This was highlighted as important in terms of understanding both the context within and beyond the institution supported. Successful projects tended to be based on solid needs assessments, and a design that recognizes the complex political economy in which the institution operates. In the governance sector, political economy is especially important. For example, enhancements in institutional capacity and performance results not only in winners, but some stakeholders lose out from changes to the status-quo, such stakeholders can therefore block reforms and capacitation efforts.

• Country ownership and leadership. High level ownership and leadership, particularly where projects seek to support reforms, are crucial. Political blocks can derail projects.  Aligning ISPs to broader capacity development or sector strategies supports ownership. Similarly, ownership and leadership at working levels supports project success – including in design and procurement stages.

• The realism of the time period. Agencies are often unrealistic about the time it takes to develop institutional capacity. They want to demonstrate results quickly, and while outputs may be delivered quickly, capacity development is a longer term process. Short term projects with transformational objectives frequently fail to live up to expectations. Further, in the rush to deliver the outputs on time, longer term sustainability concerns can be deprioritized. This problem is exacerbated by the problem of late start-up –reducing often already limited time for implementation.

• Clarity of objectives alongside flexibility in implementation. Successful interventions are characterized by objectives which are clear to all, both at strategic and at operational level. At the same time they include a degree of flexibility both in the design and approaches available and during implementation. The review of evaluations revealed that opportunities to enhance project performance were missed because of limited flexibility in the implementation phase. Capacity development is a complex process, and institutions are constantly changing – both in terms of needs and context, supporting projects also need some flexibility to ensure they are still relevant. In the governance sector specifically, contexts and political economy and change quickly, and projects at the centre of governance reform efforts may be the first affected by political change.

• Sound monitoring and evaluation. Poor monitoring frameworks are associated with under-performance for two reasons. First, where a framework to establish a logical link between activities, outputs and outcomes, and to underwrite the theory of change behind an intervention, has not been developed, there is less assurance that the planned activities and outputs will indeed lead to the desired outcomes. Second, the main role of monitoring is to track progress and to use the information to make adjustments where necessary. This means that poor monitoring, or monitoring focused on financial accountability rather than progress towards results, strips project managers of the tools they need to identify  problems and to correct projects that are heading off course.

Main Recommendations

• Reduce fragmentation by: (i) making clear in CSPs and RISPs the level of prioritization given to this work and ensuring selectivity in which sub-sectors the Bank intends to engage in, based on analysis of country needs, the Bank’s added value compared to other partners, and complementarity with other Bank investments; (ii) designing individual projects which are focused, particularly when funds are limited, rather than fragmenting funding into small parts.

• Address the gap in the strategic framework when it comes to the role of Bank operations in supporting capacity development, beyond training, in RMCs and RECs.

• Develop technical guidance for staff and partners designing and managing projects to support institutional capacity in the governance area, which is practical and also provides criteria which can be used to inform the quality assurance process and which covers the following key areas:
  i) Context, in terms of needs and political economy.
  ii) Objective setting and appropriate indicators.
  iii) Value for money, guidance should include an idea of unit costs (for training, equipment and TA).
  iv) Preference for coordinated or integrated implementation arrangements and of using of country systems.
  V) Linkages with Bank guidance on working in fragile contexts, including

• The Bank should invest in its existing staff by developing further in-house expertise in the following key areas crucial to project design and management:
  i) Assessing country capacity gaps and needs and identifying objectives and indicators that enable measurement of progress towards capacity related objectives.
  ii) Political economy analysis and the new sub-sectors where the Bank is intending to engage, according to the TYS and GAP II.

• Re-focus on regular and substantive engagement with partners rather than periodic supervision of institutional support efforts by (i) re-assessing whether bi-annual supervision remains the most useful indicator to drive monitoring practices; and (ii) devolving overall coordination and leadership for design and implementation to field offices and regional resource centers, wherever possible

• Use institutional support projects to pilot: (i) streamlining of Bank procedures and practices, particularly the long approval process; (ii) using partner's procurement systems wherever possible and, particularly where they have been assessed as adequate for PBOs.