Evaluation Team

• Task Manager: Khalid Samir, Evaluation Officer (OPEV)
• ECG assigned team

Objective

The review stated that, “Nearly all the organizations covered have made significant progress towards harmonization. On average, their practices are now materially consistent with 69% of the good practice harmonization standards, well above the 59% for the good practice standards in 2004 and 39% in 2002”. The report also stated that, “although it’ – AfDB - ‘improved its consistency rating for the substantially unchanged standards from 39% in 2002 to 65% in 2010 (on an adoption and application basis), it has not yet adopted any of the new or substantially changed standards. AfDB may not have devoted sufficient attention to the private sector GPS, since private sector operations accounted for only 26% of its 2008 investments.”

Main Findings

• The review found that all Multilateral Development Banks (MDBs) covered in this assessment made significant progress towards harmonization of their evaluation mandates, policies, and practices. When taken on average, MBDs are now at 69% consistency with the good practice harmonization standards, which is well above the 59% achieved in the previous (2nd) benchmarking exercise.2. Notwithstanding the previous, MDBs have not yet attained the desired level of harmonization as per the mandate of the Development Committee Task Force. This is evident in the fact that compliance score of some of member MDBs are below acceptable levels. In addition, MDBs continue to use the terminology employed by the ECG’s GPS for their rating categories, at the time when the content of those categories is sometimes different. Finally, MDBs have not yet reached a consensus on the experimental standards introduced in the third edition of the GPS.

• Notwithstanding the previous, MDBs have not yet attained the desired level of harmonization as per the mandate of the Development Committee Task Force. This is evident in the fact that compliance score of some of member MDBs are below acceptable levels. In addition, MDBs continue to use the terminology employed by the ECG’s GPS for their rating categories, at the time when the content of those categories is sometimes different. Finally, MDBs have not yet reached a consensus on the experimental standards introduced in the third edition of the GPS.

• The low level of compliance is caused mainly by two factors. Firstly, some MDBs did not agree some of the standards simply because they did not agree with those standards. Secondly, some MDBs did not undertake the necessary effort to incorporate the new standards into their evaluation practice, or retrofit rating of earlier evaluations to allow comparability over time, since change naturally requires time and effort.

Main Recommendations

Recommendation(s) to the Bank:

• MDBs should ensure that the next edition of the GPS build upon the progress already made in harmonizing their standards but should further clarify and strengthen the standards used in the GPS. They should not accept practices that can lead to misleading ratings on development outcomes, investment profitability, work quality and/or Role and Contribution (additionality). 

• MDBs, lagging in terms of their compliance scores, should commit to bring their guidelines and practices into line with the GPS by the next benchmarking review. Moreover, the next benchmarking review should allow sufficient time to assess whether a reasonably sized random sample of each member's evaluation reports is consistent with the GPS 

• The AfDB has to show stronger commitment in bringing its private sector evaluation practice into line with the latest edition of the GPS (fourth edition). Furthermore, AfDB (via OPEV) should continue its engagement and activities as member of the ECG to help efforts aiming at further clarification of the standards included in the GPS.

• AfDB’s guidelines for evaluation of private sector operations, which were issued in 2004 and 2005, are outdated. There is a strong need to update the Bank’s guidelines5 for better alignment with the latest edition of the GPS6. This is necessary for the achievement of two objectives, the first is better harmonization with other partner MDBs, and the second is ensuring that the Bank’s evaluations yield most reliable and most useful results.