Synthesis Report on the validation of the 2017 Project Completion Reports (PCRs)

The African Development Bank Group (AfDB or “the Bank”) undertakes self-evaluations of its completed public sector projects through Project Completion Reports (PCRs) prepared by the appropriate operations department at the end of the project cycle. Independent Development Evaluation (IDEV) subsequently reviews the PCRs on an annual basis and produces a Project Completion Report Evaluation Note (PCREN) for each PCR as well as a synthesis report on the yearly PCREN cohort. Independent validation of Management’s self-assessments of private sector operations (called Extended Supervision Reports) is undertaken as a separate exercise.

This report synthesizes findings from the review of 88 PCRs completed in 2017. The objectives of this assignment included: (i) assessing the quality and validating the performance of each of the 88 projects covered in the PCRs; (ii) assisting AfDB Management and staff to improve the quality of the PCR system; and (iii) contributing to IDEV’s Evaluation Results Database (EVRD) on project performance and PCR quality.

The synthesis report finds the following:

i) both PCRs and the independent review found the relevance of the development objectives for the projects in the portfolio to be highly satisfactory;

ii) on average, the PCRs rated development effectiveness as satisfactory whereas the PCRENs found it to be less than satisfactory;

iii) the PCRs on average rated efficiency as satisfactory, while the review found it to be less than satisfactory;

iv) the sustainability of the reviewed projects was found, on average, to be satisfactory by both the PCR and PCRENs;

v) in the case of the Bank’s performance, the review rated it lower than the PCRs. In general, it was observed that the Bank’s performance was systematically rated satisfactory or above in the PCRs, even when the project had major implementation issues;

vi) in most cases, the rating of the borrower’s performance in the PCRs was neutral and often evaluated as satisfactory, even in cases where the borrower’s performance was obviously poor;

vii) the average performance of other stakeholders was found to be satisfactory by both the PCRs and PCRENs;

viii) the review found that the M&E results framework was often inadequate and there were issues with inadequate baseline data, inappropriate indicators, as well as weak implementation and utilization of the M&E system; and

ix) the review found the quality of PCRs to be uneven

The recommendations outlined in the report are focused on project preparation and appraisal, project supervision/implementation, and improvement to evaluate projects. Some of the main recommendations concern: 

1. Accuracy of project cost estimates: Appraisals should certify to the Board that the project designs and cost estimates are relevant and reliable.
2. Borrower Capacity: The project scope should be limited when capacity is weak and where there are insufficient resources for operations and maintenance. Borrower capacity should be given greater emphasis in appraisal to ensure it is adequate for the proposed project.
3. Quality of supervision reports: Supervision reports should not overly focus on check boxes but should address any major problem areas or strategic issues that may be of concern and which should be referred to higher management.
4. Restructured projects: The PCR guidelines should make it clear that the outputs and outcomes from the appraisal report logical framework need to be the basis for the PCR unless there is an official revision to the project.
5. Need for greater emphasis on design and readiness, and implementation: The quality of project preparation (or quality at entry) should be given much greater prominence under Bank and borrower performances.
6. Need for PCR validation meetings: The introduction of a formal validation meeting would be a step towards improving the quality and reducing the disconnect between self-evaluation ratings by operations staff and those by IDEV.
7. Improve the Bank document management and retrieval database: Since this kind of review is undertaken annually, it is important that a concerted effort is made to assemble all the needed documentation prior to the next round.

The Synthesis Report of 2017 PCRs was presented to the Bank’s Board of Directors together with the Synthesis Report of 2016 PCRs, and IDEV also produced a Summary Note drawing together key information from the two reports.