Synthesis Report on the validation of 2016 Project Completion Reports (PCRs)

The Independent Development Evaluation (IDEV) at the African Development Bank (AfDB) reviewed the 49 public sector operations for which AfDB Management undertook a self-assessment (Project Completion Report, PCR) in 2016. IDEV produced an independent Project Completion Report Evaluation Note (PCREN) for each project as well as a synthesis report on overall PCREN results. Independent validation of Management’s self-assessments of private sector operations (called Extended Supervision Reports) is ongoing as a separate exercise.

The main objective of the evaluation process was to draw pertinent lessons and make recommendations to improve both the management of current projects as well as the design and management of future projects. The PCRs reviewed related to multi-sector projects (governance, social, finance sectors) as well as some in the infrastructure (power, transport, water supply, and sanitation), agriculture and environmental sectors. A consulting company was recruited to assist IDEV in the preparation and production of PCRENs for the 49 projects completed in 2016.

This report presents a synthesis of the 49 PCRENs, covering topics such as project performance, PCR quality, project Monitoring and Evaluation (M&E) systems, and gender sensitivity of M&E. The assessment found the performance of 38 (77.5%) of the projects to be satisfactory or highly satisfactory, although Management’s self-assessments (PCRs) tend to score project performance significantly higher than the independent validations (PCRENs).

Projects score most highly for their relevance. Under effectiveness, project outputs scored higher than outcomes, which was to be expected since outputs, in a well-designed project, would be a necessary, but not necessarily sufficient, condition to achieving outcomes. The efficiency indicator looked at the timeliness of projects, among others. It was notable that only 12 of 49 projects were completed on time and that most required quite substantial extensions. The sustainability dimension covered a number of important areas such as the sustainability of institutions, partnerships, and environmental aspects, which generally scored well. The performance of the project partners was generally considered to be satisfactory, with the highest score obtained by the Bank itself. Projects scored poorly on M&E design, implementation and usage. Inclusion of gender in project design was the weakest, and clearly requires improvement. Although generally satisfactory, many PCRs scored poorly on the provision of data and evidence to back up related conclusions.

The report includes a set of fourteen recommendations, of which the 5 most important ones are:

1. The most urgent need is to improve project M&E in the gender dimension. It is recommended that the Bank explore designing a standard M&E framework for its projects based on a single “living” document.
2. In conducting the PCRENs, IDEV often found crucial documents missing. It is crucial that the Bank defines and enforces a core set of documents that must be made available to the independent evaluation team.
3. It is strongly recommended that the Bank pay more attention to ensuring the financial sustainability of project outcomes and impacts.
4. As the results currently being reported by the PCRs are generally unconvincing, it is recommended to undertake a major review of Cost Benefit Analysis.
5. Given the relatively few projects that do not require an extension, it is recommended that the Bank undertake a review of late projects to identify generic issues that could be remedied for future projects.

The Synthesis Report of 2016 PCRs was presented to the Bank’s Board of Directors together with the Synthesis Report of 2017 PCRs, and IDEV also produced a Summary Note drawing together key information from the two reports.