Evaluation Team

This report was prepared by Mr. Zahid K. KHAN, Public Utilities Consultant under the overall supervision of Mrs. G. HALL-YIRGA, Principal Evaluation Officer, Operations Evaluation Department (OPEV), following their field mission to Ethiopia.

Objective

The thrust of this review is on the results of the AfDB assistance to the agriculture sector. The objectives of the review are to assess performance outcomes of the AfDB assistance to the agriculture sector in order to establish its record of achievements, and draw lessons and propose improvements on current and future Bank Group interventions. This review is also meant to provide a basis for future assessment of the performance and impact of the AfDB assistance to the ARD sector.

Main Lessons

• There are important processes contributing to the relevance of the ARD assistance: the Bank Group learning and improved knowledge of the country’s agricultural development potentials and constraints through investing in studies, compliance with the Bank Group vision of poverty reduction and its agriculture policy, GOE managed donor coordination process as a vehicle for project identification, and consultations with in-country stakeholders. The reliance on the GOE for project identification is consistent with the principle of country ownership provided the Bank Group undertakes its own independent appraisal consistent with its own guiding principles and comparative advantage, and engages proactively in dialogue to ensure necessary policy reforms are implemented.

• The quality studies commissioned for project identification tends to be high in studies that have a long history of academic research and/or strong GOE support. On the other hand, the quality of studies tends to be weak where the Bank Group has initiated without strong link to the research community. Anchoring on existing empirical knowledge, pooling and tapping on existing expertise, and flexibility in managing studies are essential elements for improving the project quality at design

• Several factors have hampered the implementation performance of the Bank Group: inadequate specification or understanding of rules and procedures, imbalance between project size and complexity and implementation capacity which was further weakened with the decentralization of the Government admininstration, delay in responding to emerging operational problems and taking remedial measures particularly in early phase of implementation, weakness in quality assurance of performance, unfamiliarity in complexity of issues, and inadequate utilization (or, capacity building) of the ETCO staff for providing technical and administrative support.

• Underlying factors in cost inefficiency in projects, which manifest in delays in procurements, disbursements of funds and retention of skilled manpower, lie in inadequacy in project conception and design, inflexibility in project implementation, failure to align project size to implementation capacity (or, sequencing capacity building), poor project governance (political influence in project management decisions), inadequacy in incentive (lack of reward system for proven performance), and lax in Government commitment to necessary legal covenants.

• Institutional development tends to be weak in an environment where technical support is inadequate (e.g., developing monitorable indicators for effective M&E), existing organizational arrangement is not suited to project requisites, incentive structure is weak, policy and/or legal ambiguity prevails (e.g., undefined property rights governing land and water), and community mobilization and participation is limited.

• Sustainability of project benefit at affordable cost is a function of operational efficiency as determined by financial and economic viability (efficacy and efficiency), institutional development, capacity to withstand external factors, and government commitment especially policy and legal support. Sustainability is assured where financial viability is strong, organizational arrangement is effective, institutions governing behavior permit adequate incentive and risk taking, contingency plans exist to pool risks, and policy and legal uncertainty is diminished.

Main Recommendations

Recommendations to the Bank:

• The Bank Group has taken significant actions to improve effectiveness of project performance since the mid-1990s: enhance ownership of projects, improve relevance of projects and quality at entry, and implementation performance.

• Raise the standards for ensuring relevance: Continue the current focus on promoting smallholder agriculture with poverty and food security focus. The application of compliance, alignment and confirmation rules is commendable to ensure relevance. But improve transparency in choosing projects guided by core principles and comparative advantage, establish sound economic justification for locating projects, and use analytical and financial leverage in influencing policy and institutional change to enhance project effectiveness.

• Continue investing in pre-project studies: The practice of “formulating new projects based on studies of pre-existing conditions” is an important progress and essential tool for AfDB to be pro-active in investment choice, and hence needs to be sustained with greater effort on improving the analytical content and quality assurance. Include in these studies careful analysis of past experiences, distill essential lessons to develop coherent project, assess existing institutional and implementation capacity, and understand potential sources of risk affecting project performance.

• Ensure coherence within and between projects: As long as the AfDB continues its current project approach, it is instructive to follow the holistic approach as prescribed in the Bank Group policy for agriculture and rural development (ADB/ADF, 2001) to maximize project effectiveness through ensuring essential project components are implemented simultaneously. The current agriculture support project is an example with components that include small scale irrigation (SSI), rain water harvesting (RWH), crop development and marketing, integrated ecosystem management, and capacity and institutional building. Provided the project assures other missing components such as access to roads and markets, and credit are available in project areas, there is coherence in the project components.

• This review reiterates the recommendation stated in the agriculture sector identification mission of late 2002, which called for a shift from a project approach to integrate all projects into a well defined and closely linked groad based agriculture sector support program.

• Align project size and complexity to operational capacity: It is necessary to either align project scope to implementation capacity or follow a phased approach where project size expands as implementation and institutional capacity develops.

• Build in margin for flexibility: Negotiate carefully implementation terms with minimum ambiguity, build in margin for flexibility to adjust as conditions change in the implementation phase, and delegate more authority and responsibility to the country office to exercise within the permissible margin.

• Establish effective M&E and response system: Adequate assessment of implementation capacity, instituting monitoring and evaluation system, and effective responses to changing conditions are necessary to ensure projects are executed in line with agreed objectives, timelines, and implementation plans. Improved communication, adequate supervision, and flexible but correct timely response to problems are necessary to ensure implementation of a project on track.
   
• The purpose of M&E is not only to monitor implementation performance per se, but to draw lessons and transfer the gained knowledge into improving design of future projects (or, interventions). It is not clear, for example, how much the lessons learned from past projects are incorporated in the new generation of projects. The absence of depository of past experience and the lessons learned should not be tolerated in current environment of technology information.
   
• Engage in Pro-active Policy dialogue: Donors meet regularly with the GOE to ensure harmonization and coordination. The forum provides the Bank Group to engage pro-actively in setting priority intervention areas. Given the financial leverage of the Bank and the presence of the Country Office in the country, it needs to enhance its effectiveness in influencing policy change and mobilization of resources for growth financing in partnership with the donor community. Beefing up the analytical capacity of AfDB/ETCO supported by the research and policy units of the Bank Group would contribute to enhance the profile and effectiveness of the Bank.
   
• Continuous consultations with borrowing country, donors, and project staff are also necessary to ensure project objectives, timing of implementation, and operation are not in conflict with complementary policy reforms. There are still overlapping activities that are likely to reduce the cost-effectiveness of their interventions such as the pastoral development projects (the Bank Group and the World Bank). A much greater coordination is still required at all levels (i.e. policy, strategy and program) within the donors, and between the donors and GOE.
   
• Promote Potential Best Practice: The Bank Group at times needs to differentiate projects in terms of their potential contribution to best practice with lessons transferable to other African countries. The RUFIP is one project with such potential. For such type of project, the emphasis should not only be whether the project meets their operational requirements, but to integrate baseline study, adequately specified monitoring performance, and impact evaluation on development outcome. The output of such project includes “lessons that are replicable or that can be scaled up” in other countries.

Recommendations to the Beneficiary:

• The record of GOE also shows improved policy framework for broad based rural and agricultural development, and project formulation. Similar to the record of the Bank Group, the GOE’s weakness lies in project implementation, institutional building including monitoring and evaluation, and sustainability. Hence, the recommendations specified for the Bank Group are also applicable to the GOE especially “continuing investing in pre-project studies”, “aligning project size and compexity to operational capacity”, “establishing effective M&E systems” and ‘learning from past experiences and promoting best practices.”

• In addition, consideration of the following recommendations helps to enhance the effectiveness of ARD.

• Complete missing policies: The GOE has set the right policy environment for smallholder-agriculture led rural development: macroeconomic policies, poverty reduction programs and food security strategy. Important agricultural policies are also in place that address main growth constraints: infrastructure development, technology transfer, building skilled manpower, and formation of agrarian institutions. But there are missing or incomplete policies: property rights in general and land in particular, input and financial market reforms and sequencing, private sector development, and population mobility and settlement.
   
• Develop sector-wide program that guides efficient resource allocation: Growth in agriculture has been slow and its effectiveness in improving human welfare has been weak. Greater efforts are necessary to jump start and speed agricultural growth and transformation. This calls for thorough understanding of the growth constraints of agriculture in its heterogeneous production environments and developing sector-wide investment program that recognizes differences in paths of development. With such program, more clarity is due on rules that guide investment allocation, placement of projects and resource commitments.

• Supporting studies that form the bases for project identification (e.g., the commendable river basin studies undertaken my MOWR) is commendable. The returns to informed investment decisions warrant that the GOE continues to invest in studies that enable it to identify agricultural projects and programs with high payoffs in terms of growth and poverty reduction.

• Build and enhance project implementation capacity at local level. Progress on civil service reform and human capacity building at local level needs to continue (e.g., expanding farmers’ training centers and skill upgrading of extension agents). More work is necessary on improving project governance to enhance accountability and financial fiduciary. The drive towards establishing community-based organizations promises improving service delivery and reducing transaction costs as problems associated with governance diminish over time. Promoting the private sector as partner to these community-based organizations is critical to broaden and deepen private sector development in rural Ethiopia (e.g., linking microfinance institutions to community based credit and saving associations).

• Enhance depository of lessons learned and replicating best practices: A key component of institutional building is strengthening monitoring and impact evaluation systems. It is important to ensure the right monitoring indicators are integrated into project design and monitored regularly to assess progress and address emerging implementation deficiencies. Combining monitoring with selective impact evaluation also enable to draw lessons and transfer gained knowledge into improving design of future projects (or, interventions). Absence of depository of past experiences, loss of institutional memory, and limited information sharing in current environment of information technology restrict policy debate and dialogue.