Objective

The overarching objective of the evaluation is to examine the relevance, efficiency and effectiveness of the PBL instrument as used by the Bank in regional member countries over the period 1999 – 2009. Specific attention is being given to the evaluation of: The AfDB’s institutional and policy framework for the design, appraisal and delivery of PBL: to evaluate the Bank’s formal policy framework, procedures, skills and organisational sructure for the delivery of PBL. How well does this reflect emerging international (and Bank) best practice; and is the Bank organisationally equipped to deliver best practice PBL efficiently and effectively? Policy and practice: to evaluate the Bank’s approach to the design, appraisal and delivery of PBL instruments in practice and how well this fits or deviates from Bank policy guidelines, with international best practice and country context and needs. Choice of aid instruments: to evaluate how choices are made about the use of aid instrument in the country programmes and how relevant it is to the partner country’s problems, policies and priorities Effects at country level: To assess the contribution made by the AfDB to policy dialogue, joint processes, institutional strengthening and other relevant areas where PBL is focused.

-----------

This report assesses the experience of AfDB support to PBOs in Ethiopia, based on evidence gathered both in country and at AfDB headquarters (HQ) (in this case through phone interviews) as well as through a review of the relevant literature. A bibliography is found at the end of the report. The country visit in Ethiopia took place from 14 to 25 September 20102. The team comprised of Catherine Dom and (Team Leader) and Mohammed Mussa. A list

Main Findings

• Over the evaluation period AfDB PBOs’ objectives and modalities have shifted .AfDB PBOs, initially focused on macro-economic stabilisation and structural and macro-level budget and PFM reforms, they now support decentralised basic service delivery and PFM/accountability systems. Delivery modalities shifted from conditionality-based BOP operations to joint conditionality-based GBS, to joint budget financing conditional on the Government of Ethiopia's (GOE’s) adherence to agreed principles related to the effective and fair functioning of the country’s fiscal decentralisation system. These shifts facilitated a marked increase in policy effectiveness since the PBS PBOs provide effective support to government-led implementation of strongly-owned policies based on a GOE-DP consensus, in stark contrast with the earlier PBOs and other similar operations by DPs focusing on economic and political governance issues and failing to shift GOE positions on ‘no-go zones’.

• Consensus around PBS led to more predictable aid flows although commitment to predictability is uncertain Because of the more consensual nature of the dialogue; PBS has been more reliable than earlier joint BOP/BS financing, and thus more predictable. AfDB has had an important role in improving the predictability of financing for the government budget. Even within the confines of the PBS there is not full agreement among DPs about the importance of predictability. For those believing that budget support should be used to influence political governance issues the Bank’s predictability is seen at times as a hindrance. Although this medium term predictability is not Bank-specific it might have been possible and certainly would have been desirable to start addressing it earlier on.

• The Bank has not succeeded in maximising the effectiveness of its support to focal sectors. AfDB has not participated in SWAps in sectors where it could have contributed. This is in part due to the ADF rule of origin which prevents participation in sector pooled funding arrangements. As a result AfDB has been unable to maximise the effectiveness of its support to sectors at the decentralised levels in Ethiopia as it has not been able to provide funds through other DPs’ more effective project modalities. The 25% cross-country cap prevented AfDB from increasing its PBS (PBO) funding which could have helped develop stronger links with the sector policy dialogue in AfDB’s areas of expertise (like water and rural development).

• Bank support to PFM reforms has been strengthened in the latter part of the evaluation period but is still not very effective. Through its support to the relevant component of the PBS the Bank’s corporate focus on PFM is now translated into specific programme measures and into increased operational capacity in the Ethiopia Field Office (ETFO). This is still strongly limited by insufficient staff access to capacity development opportunities, weak Bank knowledge management systems, unclear support from HQ, competition over staff time, and a lack of access to resources that would allow ETFO to contribute more to joint support processes. The current configuration of DP support to PFM reforms is not satisfactory. The Bank is actively engaged in efforts to address this, including investigating options to ‘hive off’ PFM support to make it more holistic and systemic.

• There has been progress by AfDB with the harmonisation of PBOs’, but the attribution of specific results to AfDB PBOs is not feasible. Harmonisation has significantly increased for both the design and the management of AfDB’s PBOs and transaction costs have diminished (from onerous import-related reporting requirements to specific AfDB triggers in the joint DBS policy matrix to near full harmonisation in PBS2). Due to the nature of the PBS the attribution of specific policy and institutional results to AfDB PBOs is not feasible. Although the intervention logic of the PBS PBOs is clearer than it was for earlier AfDB PBOs attribution issues are more intractable.

• This is partly due to better alignment and harmonisation. However, AfDB's inputs under the PBS operations support objectives defined by GOE, and a host of other factors strongly contributed to the same set of results. For example, it is the combination of DPs’ weight which incentivises GOE’s adherence to the PBS principles. AfDB funding is co-mingled in the joint PBS-GOE block grant transfers to sub-national governments and its other inputs are co-mingled in donors’ joint support to jointly agreed work plans hence AfDB's inputs are indistinguishable and at best, proportional to their contribution to the total

• Harmonisation has however, remained very limited in non-PBO operations due to Bank regulations and attitudes which are seen typically to continue to define the mainly bilateral and HQ-driven way in which the Bank manages its overall engagement with GOE.

• The Bank’s technical capacity to deliver PBOs and engage in multi-DP processes has increased but is still felt to fall short of what would be desirable .ETFO was strengthened in 2008, increasing staff numbers with a Country Economist in Addis Ababa. This allowed the Bank to improve the delivery of its PBOs through facilitating increased in-country interactions and fuller participation in the continuous joint DP management and GOE-DP dialogue processes around the PBS. The AfDB became more visible. For instance, the Bank was more fully engaged in fine-tuning the design of PBS2; and through the ETFO AfDB has been a reasonably effective (albeit politically non-engaged) co-chair of the PBS donor group.

• However, overall, the Bank is seen to punch below its weight considering the amount of financial resources invested and its capacity to undertake an active leadership role. The Bank is also perceived still to be highly centralised, although GOE and DPs recognised how the strengthening of in country capacity through the opening of the country office had already made a big difference in the way the Bank manages and is perceived to manage its PBOs

• There is a sense among other DPs that AfDB is not a strong partner at the high level dialogue (DAG/HLF). However, there is no consensus about what a ‘special role’ for AfDB might look like. In the first place it is not clear whether this special role should or should not encompass political governance issues due to AfDB’s lack of influence on and possibly a limited understanding of political developments. The fact that AfDB does not take sides is seen by GOE as one of its comparative advantages, and AfDB argues that there are other channels for discussing governance issues. Among bilateral DPs concerned with political governance issues there is no consensus. For some, AfDB should be able to send strong messages to GOE; for others, on the contrary, its special role should be to find less confrontational ways of addressing issues of common concern.

• Consensus around PBS facilitated higher levels of external financing. However, for AfDB this was limited by institutional constraints .The PBS has been highly efficient in delivering the external financing that Ethiopia needs to reach the MDGs. It has channelled large-scale funding more reliably than the earlier AfDB PBOs, and funding has gone directly to the decentralised levels in charge of service delivery which other aid modalities have much greater difficulties in reaching. There has been a doubling of the volume of AfDB resources allocated to PBOs between SAL2 and PBS2 and this will be further increased as additional financing for PBS2 has recently been approved. However, the 25% cap on cross-country PBO resources has acted as an exogenous break on AfDB’s capacity to raise more resources for PBS2 as requested by GOE. Arguably, this has limited the effectiveness of AfDB funding in so far as these funds have been used or are earmarked for less effective modalities/uses. The cap is likely to play a similar ‘artificial break’ role in future

• ‘Politics’ trumps aid effectiveness considerations and this has affected AfDB PBOs. Issues of aid effectiveness, particularly around the role of budget support, have been continuously trumped by political governance concerns since 2004/05. The focus on political governance issues has undermined efforts by DPs with the relevant comparative advantage (such as AfDB, IDA and IMF) to take up the growth agenda, as it has been difficult to assess the extent of agreement between GOE and DPs on policy frameworks related to economic development

• PBOS have become more focused and selective much of this progress came from the Bank’s decisions to join other DPs’ programme design .Since 1999 the Bank has made significant progress with greater focus and selectivity of its PBO objectives and better alignment with GOE priorities. The distinct shift in the nature of the PBO conditionality framework also arose from the Bank’s decision to join the PBS. The Bank has been able to adapt its corporate framework for Direct Budget Support Lending (DBSL) to join and therefore support other DPs. The AfDB showed flexibility in adapting its DBSL instrument in response to local circumstances so that it was able to join the PBS, even though the programme focal areas and modalities were outside its comfort zone.

• PCRs have been completed too late to provide useful lessons for PBOs .Earlier PBO designs were not based on a well-thought-out intervention logic and failed adequately to identify risks. Project Completion Reports (PCRs) for the earlier PBOs were prepared too late to be of use. There has been progress on both fronts with the PBS PBOs, and although the weak links with sector processes still limit monitoring, this is expected to improve with the more inclusive PBS2 design and strengthening of monitoring capacity. The M&E framework of the PBOs has been gradually strengthened, but there continue to be misguided concerns about attributing the PBS achievements. These in turn muddle the post-implementation assessments of the effectiveness of the Bank’s PBOs as merely contributing to joint processes. The prime logic is still to want to attribute results to AfDB inputs/influence; merely contributing is seen as ‘not good enough’ by the AfDB Board.

Main Lessons

Main Recommendations

Recommendation(s) to the Borrower:

• For the Bank Group: Greater clarity is needed with regard to the 25% cross-country cap on PBO resources and its implications for the allocation of PBO resources to the Ethiopia country programme. This would help improve medium-term predictability of Bank’s PBOs.

• For the Bank Group: CSPs should discuss issues of aid effectiveness in the country context and make clear the rationale for the specific mix of instruments envisaged

• In complex political contexts like that of Ethiopia the Bank needs to have stronger in-country all-round governance capacity to avoid being hostage to circumstances it imperfectly grasps. This would also enable it to engage with other DPs on an equal footing even if the Bank wishes to remain different from bilateral DPs. This would strengthen CSP preparation, and PBO planning and delivery

• For the GOE: More support of various kinds is needed for ETFO staff to be able to engage fully with the PBOs and in high level dialogue with other DPs. Support should be tailored to enable greater sector involvement in PBOs and more time for ETFO staff to engage in technical analyses (or commission them), and dialogue – focusing on priority areas clearly identified for the medium term. Examples of this include the timely recruitment of local expertise in priority sectors e.g. in watsan, access to more consultancy resources to be locally prioritised, capacity development opportunities in e.g. PFM, less time on administration.

• For the GOE: Ensuring continuity and seniority together with greater delegation of authority at Resident Representative level would go a long way in strengthening the Bank’s position vis-à-vis other DPs. It would also ensure that they see the Bank as having ‘clout’ and better punching to its weight

Recommendation(s) to the Bank:

• AfDB should assess how the cap constrains its aid effectiveness and/or limits the scope of its country programmes, and consider ways of addressing this. In countries like Ethiopia the Bank’s project lending instruments are inadequate for a range of important activities, whilst one clear value-added is the provision of large-scale budget financing more reliably than by bilateral DPs because the Bank’s funding does not come with political conditionality. The cap limits the Bank’s potential to fully exploit this value-added.

• Greater clarity on the Bank’s comparative advantages/disadvantages is needed as a basis for a better focused medium-term strategy, taking account of the quite clear views of DPs and GOE about this31. The Bank should discuss more candidly these issues and its strategy (including the envisaged PBO role and focus) with other DPs, in addition to its dialogue with GOE. Formal consultation processes in the course of the CSP preparation may not be sufficient. It is also important that ETFO institutional memory and robust understanding of the context be fully tapped

• For the Bank Group: PCRs for the PBOs should be prepared in a more timely manner, and a process should be developed/strengthened to ensure that they feed into subsequent operations. In the context of joint operations as is the case in Ethiopia, this process should include a discussion of PCR main findings and lessons identified with other DPs and GOE. Similarly, the Bank’s relevant evaluations should be discussed for the same reason and in a similar way