Independent Development Evaluation (IDEV) has conducted an evaluation of the Self-evaluation Systems and Processes (SESP) of the African Development Bank (AfDB or ‘the Bank’), covering the period from 2013 to 2018. SESP are commonly defined as the assessment made of a project, country/regional program, or policy/theme by the entity engaged in the activity. The AfDB’s SESP pursue three main outcomes: performance, accountability and learning, and the main SESP tools are implementation progress reports, mid-term reviews, and completion reports. The evaluation assessed the functioning of the SESP along three main dimensions: relevance, effectiveness and efficiency. It tries to answer the question: “Do the SESP support performance management, accountability, and learning at the Bank?”, and two underlying sub-questions: (i) how well are the SESP performing? and (ii) to what extent are the SESP impacting the quality of development results?
This evaluation builds on previous IDEV evaluative work assessing the processes for Bank Group operations, notably: (i) their quality at entry, and (ii) the quality of supervision and at exit, as well as a “chapeau” report, the Evaluation of Quality Assurance across the Project Cycle (2012–17), which consolidates the different findings, including on the compliance with the Bank’s environmental and social safeguards. In response, Management prepared a Quality Assurance Implementation Action Plan, covering five areas of reform: (i) reinforcing the knowledge of operations staff; (ii) strengthening project preparation; (iii) refining the assurance framework for quality at entry; (iv) sharpened focus on delivery and results; and (v) improving planning, budgeting and information).
This evaluation of the Bank’s SESP completes IDEV’s assessment of the Bank’s quality assurance framework. Together with Management’s own diagnosis, it forms the basis for the improvement of the SESP and will potentially set the stage for institutionalizing “best practices” in a consistent way, positioning the AfDB on a par with comparator agencies.
The evaluation found that the Bank’s SESP lay out strong standards and procedures underpinning their functioning, as well as a cogent articulation with the independent evaluation function carried out by IDEV. However the SESP were found to be constrained by weaknesses in the application of the established procedures, standards and norms, by their ratings methodology and structure, and the way this is applied to Project Completion Reports.
Based on its findings, the evaluation report includes the following recommendations:
- Review the AfDB’s results measurement framework and evaluation frameworks across the project cycle to ensure (i) internal consistency throughout the AfDB’s results measurement and reporting system from operation to corporate level, and (ii) that there is harmonization between sovereign (SO) and non-sovereign operations (NSO). This measure would complement the ongoing revision being carried out by the NSO and Private Sector Support Department.
- Review and revise, in close collaboration with IDEV, the Project Completion Report/Extended Supervision Report ratings methodology in use, including the ratings scale and guidelines, to improve the reliability of the ratings system for all operations, and to better align SO and NSO.
- Enhance the accountability for SESP products by developing appropriate indicators and explicitly incorporating these indicators in the AfDB’s top-level corporate KPIs, in VPs’ and Directors’ performance agreements, and staff performance discussions. The focus should be on delivery, timeliness, quality, usefulness, and proactivity, encouraging candor in flagging and addressing emerging issues.
- In the context of the AfDB’s rightsizing exercise, review the SO and NSO staffing levels for M&E and quality assurance activities to ensure that the self-evaluation function is appropriately staffed and resourced.
- Enhance compliance with corporate self-evaluation reporting requirements by clarifying the roles, procedures, frequency, and reporting requirements for supervision, implementation support, and completion. In addition, work with IDEV on an appropriate and realistic timeline for timely PCR/XSR validation to inform the Annual Development Effectiveness Report on AfDB-funded interventions.
- Enhance learning through SESPs by improving (i) the quality and accessibility of SESP products, and (ii) task managers’ understanding and acceptance of SESP as learning tools rather than just as accountability tools.
Evaluation Task Manager: Foday Turay - Chief Evaluation Officer
