Evaluation Team

This report was prepared by Mr. P-A. ROCHON, Principal Evaluation Officer, OPEV.

Objective

The objective of the CAE is to assess the development effectiveness of the Bank's intervention by evaluating the contribution of the Bank's assistance towards meeting the expected outcomes in the Government's Strategic Document, the Tanzania Poverty Reduction Strategy, and eventually in meeting the Millennium Development Goals. The CAE also draws relevant lessons learned from experience over the period in order to strengthen future Bank policies and procedures and improve the quality of Bank operations.

Main Findings

• Analysis relating to constraints to sustainable growth and poverty reduction in the three CSPs failed to make sufficient reference to Zanzibar. The increasing importance of Zanzibar in Bank interventions could signify greater risks regarding the achievement of outcomes, performance and quality of the Bank’s overall portfolio in Tanzania considering that Zanzibar has not adhered to key crosscutting issues such as Gender and that Governance issues in Zanzibar are of concern among the donor community. Given Zanzibar’s resource limitations, along with its distinct social and economic characteristics, there is a need for additional attention to be paid to Zanzibar when strategically planning the Bank’s activities in Tanzania

• Overall, effort to limit the number sectors supporting the Bank Group strategy had limited success. Sectoral focus in Bank’s programmed sectors was not delivered as planned. There was also limited selectivity based the previous strategy. Each CSP appears a self-contained strategy with limited connections to either its predecessor or its successor strategy.

• With regards to the Bank’s comparative advantage the Bank seems to marginalize itself in spite of its significant interventions in certain sectors. For example, despite high level of involvement in Transport (25% of total operations in Tanzania during the three CSP program period), the Bank did not engage pro-actively in aid coordination and advising in institutional streamlining and overall sector reforms. The Bank has yet to fully capitalize on comparative advantage in Transport, as perceived by the GoT and other donors. A similar situation exists with regards to Education.

• The mix of lending and non-lending instruments in the CSP was key to achieving synergy among various Bank endeavours during the programming period. Each CSP delivered assistance composed mostly of project loans as well as grants, and policy-based operations. There is no clear explanation in the CSPs of how projects, grants, PBLs, etc., are to work together towards the goal of poverty reduction. In addition, as the mix of lending and non-lending instruments in the CSPs was modified during implementation, it is unlikely that synergy among various Bank endeavours during the programming period was achieved as planned. With regards to porfolio size, as the average loan outside PBLs was less than 10 million UA, these required significant supervision effort which may in part explain the aging portfolio. Out of the 34 operations, 8 have been completed (PBLs and studies). None of the sectoral projects has been completed over the three CSP cycle period of 9 years. Completed and ongoing operations since 1996 have taken, on average, 14 months to become effective

• The Bank has not participated a Sector Wide Approach in Health, nor in Education (notably vocational training, adult and non-formal education). Also, there is a problem with possible duplication of efforts especially regarding construction of maternal and child health units in some areas due to the Bank portfolio not having been included inside the sector wide planning approach.

• None of the CSPs were supported by a set of medium-term performance indicators as well as a sound monitoring and evaluation system to help determine the effectiveness of Bank’s assistance program. Though CSPs presented acceptable longer-term indicators crucial to judge the success of the government’s poverty reduction strategy, they did not provide monitorable medium-term outcome indicators. It is important that in future such performance indicators be identified. The sources and the data and the selected indicators for project monitoring and benefit measurement were not clearly identified and specified in the appraisal reports. Similarly, the collection of information by the Borrower was not covered by appropriate conditions in the Loan Agreements and regularly monitored by the Bank. The lack of a sound monitoring and evaluation system in the CSPs was a design defect in the development of the CSP guidelines which will hopefully be remedied with the new RBCSPs. This problem was compounded by the absence of compatible monitoring and evaluation systems within the GoT. Poverty issues had no clear objectives and indicators that could facilitate monitoring and evaluation.

• The evaluation also shows that ESW plays an important role in formulating and supporting reform programs that underline Bank lending, particularly in improving institutional development. In this regard, diagnostic work such as sector work, would help Bank to position itself when it has significant interventions in sectors where it has not yet fully capitalized on its comparative advantage.

Main Recommendations

Recommendation(s) to the Bank:

• At the outset of future CSPs, selectivity, as well as the mix of lending and non-lending instruments used to achieve synergy among various Bank endeavours during the programming period should be a key consideration. Operations should pursue efforts to limit the number of sectors supporting the Bank Group strategy with a view to increasing the average loan amount. CSPs should also clearly demonstrate how projects, grants, PBLs, investment projects, etc., are to work together towards the goal of poverty reduction 

• Economic Sector Work (ESW) is key to building the information/knowledge base to underpin the Bank’s lending operations and to inform the policy dialogue process. ESW should become a prerequisite to enhancing the quality of Bank interventions. It is imperative that the Bank Group initiate Economic Sector Work, which could be carried out solely or in conjunction with other development partners (World Bank/IFAD)
   
• Bank should seek to leverage funding and augment its own resources devoted to ESW (i.e. surveys, studies, programme design, etc.) for sector interventions. In addition, the next CSP should include robust analysis of Zanzibar to provide a better understanding of the island’s needs and the island’s weight in the Bank’s overall strategy. It should include risk analysis of further investing in Zanzibar.

• With regards to PBLs better partner with BWI in setting the tone of programme design and preparation. Special attention should be paid to policy issues linked to ongoing and/or planned sectoral interventions in the CSP. The Bank should increase and raise the level of its dialogue with the government to remove a persisting perception that the Bank is playing second fiddle to other donors (notably the World Bank). 

• The Bank should pursue its Multi-donor Budget Support (MDBS) and participate in SWAPs with other donors while ensuring its intellectual contribution to these types of interventions is well founded in ESW. For example, to further enhance Gender mainstreaming, the CSP should adopt recommendations from the recent Tanzania – Multi-Sector Country Gender Profile, notably with regards to Bank CSP development, budget support initiatives, Policy Based Lending Programmes, and SWAPs.
   
• Efficiency is key to Bank programme delivery. A special meeting between the Tanzania Country Director and ADB Project Coordinators in Tanzania (along with Ministry of Finance) be held in the next three months in order to hear first-hand and address the numerous efficiency issues. This should be expressly mentioned in the next CSP. In particular, to minimize implementation delays the following measures should be implemented

• Disbursement schedules should reflect Tanzania’s historical experience for each sector with an appropriate (modest) "improvement factor". Major departures from historical patterns would then require special justification.

• Training in the Bank’s procurement and disbursement procedures should be made an integral part of project processing and should be offered to officials who require such training, for each project ideally before approval. Undertaking project launching missions for all projects will also help in getting projects off to a good start.
   
• Loan effectiveness delays can be minimized by requiring the fulfilment of substantive conditions before appraisal or on Board approval. Effectiveness conditions would then be limited primarily to legal requirements.
   
• PCRs are not carried out in time or systematically and thus there are no useful lessons learned from PCRs. This can be remedied by carrying out PCRs without delays so that good lessons learned from what are considered success stories can be used elsewhere on timely basis. CPPR also need to be carried out on a regular basis. There is also need to ensure adequate budget for preparation of PCRs. The Country Director should draw an action plan to be current on PCRs and CPPRs.