Evaluation Team

The evaluation was undertaken by Stephen Jones and Ann Bartholomew, consultants, under the supervision of Joanne Asquith (as Chief Evaluation Officr, OPEV), with contributions from Colin Kirk (as Director, OPEV) and Odile Keller (Division Manager).

Objective

This evaluation assesses the African Development Bank’s use of policy-based operations (PBOs) over the period 1999-2009. It examines how effectively PBOs have been used for the achievement of nationally owned development objectives. The main focus of the evaluation is on the Bank’s internal processes (policy and guidelines) for designing and delivering PBOs, and how these have translated into decisions about how PBOs are used in different country settings.

Main Recommendations

Recommendation(s) to the Bank:

• the Bank needs to define more clearly and authoritatively the role that PBOs have in advancing the Bank’s overall mandate and objectives, and its engagement at country level. When the Bank participates in joint budget support operations, as it does in the majority of cases, the specific value added from the Bank’s participation needs to be identified, as well as potential synergies with other Bank activities, notably investment lending and support to capacity development.

• Second, the Bank needs to produce a (single) comprehensive policy and supporting guidance on PBOs (replacing, where necessary, existing policies and guidance), building on the 2004 policy on Direct Budget Support Lending. The policy and guidance should clearly state the intervention logic to be used in the statement of the objectives and design of the PBOs; and should draw on lessons and best practice in defining objectives, conditionalities, and other design aspects such as tranching.

• the organisational and management implications of the new policy will need to be addressed to support its effective implementation. This will include (a) clarifying the role of different Bank departments and organisational units (including, in particular, OSGE and Field Offices) in the design and management of PBOs; (b) building capacity within the Bank to ensure that organisational changes are effectively implemented; © developing procedures specifically tailored to the design, appraisal, and management requirements of PBOs, to replace the use of generic project preparation procedures; and (iv) improving information systems to enable the performance and results of PBOs to be more effectively monitored and lessons identified.

• The Bank should take a decision about how the PBO instrument is to be used to support the Bank’s wider strategic objectives and the needs of its RMC clients. To the extent that the Bank decides that PBOs should be used to strengthen engagement in policy dialogue, rather than just a financing mechanism, the Bank needs to build its capacity and develop its policies and procedures to fulfil this role more appropriately. This would require stronger technical and policy capacity, particularly in Field Offices; a deeper engagement in economic and sector work; and departmental and individual incentives to encourage cross-sectoral working.

• Guidance for the design of PBOs and identification of results should be based on a more fully developed model of their intervention logic; and the design framework should link the levels of financing to the scale of results. The Bank’s results reporting should be based on the PBO’s contribution to government expenditure; identify additional value added through the Bank’s participation in policy dialogue and complementary support; and clarify outstanding issues about fiduciary risk management for PBOs. This may involve use of political economy analysis as an integral part of the design and monitoring of PBOs.

• Existing PBO policies and guidance should be consolidated into a single policy. This may involve consolidation of existing instruments into a single instrument based on the current DBSL instrument. The Bank should review the use of the term of “policy-based operation” to refer to all types of budget support operations, as the term has the inappropriate connotation that these operations seek to bring about policy reform, even if they do not include policy dialogue.

• The Bank should identify potential synergies between its engagement in general budget support and other parts of the Bank’s programme, especially investment operations, as part of the preparation and review of Country Strategy Papers, as well as during the design of individual operations.
   
• To the extent that the Bank identifies potential synergies and contributions it can make through policy dialogue or complementary activities—including by chairing joint budget support groups—these should be explicitly built into the design of PBOs. They should also be supported with appropriate capacity and resources (especially in Field Offices) to ensure that they are effective.

• As part of the process of developing the Bank’s new policy on PBOs, each sectoral area of the Bank should review its scope for using sectoral PBOs, contributing to multisectoral PBOs, and more effectively participating in SWAps. This review should include a discussion of specific issues relating to PBOs for that sector, while recognising that the scope for sectoral PBOs depends on both the preferences of clients and the extent to which sectoral (rather than general) budget support programmes are able to address key constraints on sectoral performance, which may require cross-sectoral action.

• The Bank should examine how policies and procedures (especially relating to ADF) can be adapted to allow a more programmatic mediumterm approach to budget support. This could be modelled on the World Bank’s programmatic approach to Development Policy Operations, based on the use of single-tranche operations within a multi-year framework.

• The Bank should develop, fully document, and provide comprehensive training for staff in a set of procedures and timetables that are specifically tailored to the needs of PBOs and take account lessons from the Bank’s rapid response to the urgent needs of RMCs during the international financial crisis. These procedures should be more flexible than those currently set out in the Appraisal of Projects Processing Schedule, while encouraging greater attention to analysis and more effective cross-sectoral teamwork than currently takes place.

• The Bank should review and substantially strengthen its information systems and procedures for PBOs, which currently (in SAP) contain significant weaknesses and apparent inconsistencies, which constrain effective evaluation and monitoring of the Bank’s PBO portfolio as a whole.