Evaluation Team

This report was prepared by Albert-Enéas Gakusi, Principal Evaluation Officer and Team Leader; M. P. Madhusoodhanan, Long Term Consultant; and Marc Mitchell, Short Term Consultant and Lecturer on International Health at Harvard School of Public Health, Boston. Matthew Hodge, Short Term Consultant, undertook the health project design quality analysis. Getinet W. Giorgis, former Director of Operations Evaluation Department (OPEV), provided guidance since the initial phase of the review in July 2005 until his retirement at the end of January 2006. Douglas Barnett, Acting Director took over since February 2006 and gave it a new momentum. Mohamed Manai, Chief Evaluation Officer at OPEV provided advice on methodological issues and the organisation of the report. The report received valuable comments from Osvaldo Feinstein on the approach paper and a thorough peer review and advices from Edward Elmendorf on the draft report. Detlev Puetz, Principal Evaluation Officer at OPEV read the final draft report and provided  ditorial comments.

Objective

This review aims to assess how ffective are the Bank policies, strategies and interventions in the health sector in order to draw lessons for the future. The review was carried out through an extensive review of Bank documents dealing with policy, project and post-evaluations; relevant literature on the health sector in Africa; analysis of Bank project-related data; consultations with Bank health staff and managers in charge of policy formulation and project and programme implementation.

Main Findings

• The burden of disease in Africa remains the highest in the world and is predominantly characterised by infectious diseases that could be either prevented or cured in a cost-effective way. Since the 1980s, the health situation has faced more formidable challenges including emergence of HIV/AIDS and chloroquine resistant malaria strains, increase in noncommunicable diseases, continuing population growth, and poor economic performance. The health system is very ill-equipped and the financing is inadequate to cope with the many health challenges.

• Despite the Bank commitment to MDGs, the health sector remained under-funded with a decreasing proportion of the health sector from 4.1 per cent of the total Bank portfolio during 1996-2000 period to 3.5 per cent during 2001-2005 period. For the period under review the Bank has spent only 3.4 per cent of its development assistance on health. The Bank’s share of the total donor funding for heath and population sectors in Africa for 1990- 2004 period represents only 5.4 per cent and this limited resource is thinly spread across RMCs in small projects, which have, at best, generated localised impacts.65 Further, the portfolio analysis shows also that the Bank neglected the population sub-sector including reproductive health and nutrition while these constitute some of the biggest challenges faced by African countries.

• The Bank interventions have been mainly oriented to primary health care, the most appropriate health care strategy recommended since the Alma-Ata Conference in 1978 for its universal access, equity, community participation, and inter-sectoral approaches to health, although difficult to implement (WHO 1990; Rainhorn and Burnier 2001; Schieber et al. 2006). Towards this end, the Bank interventions have responded to the policies and priorities of RMCs guided by recommendations of specialised international organisations, notably WHO, UNAIDS, UNFPA, and UNICEF. Nevertheless, health sector receives a small share of the total Bank portfolio in spite of increasing recognition that human capital is critical for economic development and poverty reduction. The portfolio analysis shows also that the Bank neglected the population sub-sector including reproductive health and nutrition while these constitute the biggest challenges faced by African countries. Further, the Bank has not yet undertaken significant actions to support the private sector which represents nearly 50 per cent of the total health spending.

• The Bank has diversified its health portfolio from infrastructure to health sector reforms and capacity building in tune with other international development organisations. Infrastructure category financing declined from an average of 79 per cent during the first policy period 1987-1996 to an average of 64 per cent in the second policy period 1996 up to now. This percentage is about 50 per cent since 2003.

• Available data show that the Bank continues to have difficulties in improving the quality of its health portfolio in terms of implementation and outcome performance contrary to what is observed about other development partners like the World Bank, UNDP or DFID (World Bank 1999; UNDP 2000). Only 50 per cent of Bank health projects have a satisfactory overall implementation performance. While the number of ‘problematic projects’ has declined since 1998, compliance with Bank rules for procurement, adherence to implementation schedules and quality of activities and works have registered decline despite several portfolio improvement measures. Only 42 per cent of completed health projects with PCRs achieved satisfactory performance, with Bank performance satisfactory in 50 per cent of cases and Borrower performance satisfactory only 25 per cent of the time. Among outcome indicators, relevance and achievement of objectives (50 per cent) did better than institutional development (42 per cent) and sustainability (33 per cent). Although Bank projects provided modern equipment for health facilities, in many cases, shortage of trained health personnel and inadequate provisions for recurrent expenses undermined project sustainability. Facilities have generally increased physical accessibility and service utilisation, but user fees have deterred the poor from accessing the facilities.

• Though the Bank’s achievements in the health sector leave much to be desired, it has the potential to become a far more important and positive force in improving Africa’s health. At present, however, this potential is largely unrealised due to the low priority given to health investment, a reactive rather than proactive approach to project identification and funding, and a staffing and incentive structure that emphasises administrative procedures and project approvals rather than effective implementation and measurable results on the ground. These issues present significant obstacles to the Bank in realising its potential to influence health outcomes in Africa. While challenging, the needed transformation are well within the Bank’s reach, given the prevailing support environment for change, the institution’s real advantages of being an African institution and access to key decision makers, potential synergy from working in many different sectors, and expertise in infrastructure development.

Main Lessons

• Weak use of economic and sector analysis (done either by the Bank or by other ,institutions) prevents the Bank from producing qualitatively superior project designs, carrying out comprehensive appraisals and effective implementations, adversely affecting project results

•  Inadequate attention given to preparation and appraisal lead to unreasonable estimates of time and cost required for project implementation, inadequate attention to risk factors, eventually jeopardising implementation and outcome performance. 

• Lack of adequate monitoring and evaluation system prevents the Bank and RMCs from learning from health investments and from being fully accountable towards stakeholders and African populations

• The participation of different stakeholders – including medical personnel, beneficiary population and NGOs – in the design of health projects ensures that the technical and cultural aspects are adequately taken into account, and enables enhanced utilisation and sustainability of the facilities

• Government’s commitment and quality of institutions are crucial for making successful equitable investments from national revenue and international assistance.

• Lack of adequate incentives in terms of accommodation and transportation facilities makes it difficult to attract and retain qualified personnel in rural areas and to find a solution to human resources deficit and regional imbalances. 

• From the Bank’s experience of funding the private Abuja International Diagnostic Center, the lesson learned is that when governments are part of the private project implementation arrangements, there is a need to adopt more stringent forms of legal obligations

Main Recommendations

Recommendation(s) to the Bank:

• Health Policy and Policy Guidelines: It is recommended that the Bank revise its current health sector policy, strategies, and guidelines to update them and make them more operationally focused and identify areas where it can have the greatest impact.66 In order to ,identify these areas, the Bank needs to undertake an intensive consultation process with its donor partners, financiers and clients before coming to final conclusions and ways forward. The policy development process should be as inclusive as possible with involvement of health specialists at the country/regional levels and at the Bank. The Bank should make use of health policy guidelines mandatory for project identification, preparation and appraisal to ensure that the projects and programmes financed by the Bank reflect the operational ,priorities articulated in the policy. Internal process should ensure accountability for use of guidelines, and responsibility for managers for this result. The health sector policy guidelines should contain detailed guidance for performing technical aspects in project design and management.

• Quality of the Projects: The Bank needs to make substantial efforts to improve the quality of its health sector projects at the design and appraisal stages through adequate assessment of the capacity within the Bank and in RMCs to monitor and implement projects. In order to facilitate this, the Bank needs to determine: (a) appropriate skills-mix required for its health interventions; and (b) a realistic workload for its health experts allowing them to improve the quality of project appraisal based on appropriate analysis of the health sector including health demand, institutional capacity, and risks assessment. To this end, the Bank management should carry out an assessment of the existing technical skills profile of its health professionals by sub-areas of health in light of renewed health policy directions.

• Participation in SWAp: The Bank should actively participate in and even initiate Sector-Wide Approaches (SWAp) in the health sector. To facilitate this process, the Bank needs to draw effectively upon good Economic and Sector Work carried out by the Bank or by others institutions. Incentives need to be established to ensure that the Bank staff use suitable analysis. Country readiness for engaging in SWAp must be analysed

• Policy Dialogue, Advocacy, and Partnership: On the basis of new health policy and guidelines, the Bank should take a lead in selected policy dialogue with RMCs supported by detailed Economic and Sector Analysis. Staff should be rewarded for contributing to effectively promoting evidence-based national policies that have a positive impact on the health status of the population. Also, the Bank should enhance its partnership with other Development Partners in the health sector based on the strengths of each participating organisation.

• Priority Investments: The Bank should augment its assistance to selected areas of strategic importance with high priority to training health personnel, integrated health system, and in supporting reproductive health on account of their importance, inadequate attention by donors, as well as the potential to produce substantial demonstrable results. The Bank should formulate a strategy for health human resources development, as the existing ones do not provide sufficient operational direction. To reduce the burden on clients, the strategy should be harmonised among MDBs

• Private Sector Financing: The Bank should undertake feasibility studies aiming at exploring possibilities to effectively engage in private sector financing. Based on the Abuja International Diagnostic Centre experience, the Bank should ensure that all financial, infrastructural, legal and other institutional pre-requisites are in place prior to any project support to avoid risky investments. The new Bank health policy should define how to encourage the public-private partnership building on respective comparative advantages of each sector.

• Resource Commitment: It is recommended that the Bank look into the possibility of increasing funding support for health sector interventions. It should consider increasing the average size of health sector loans as administrative burden, especially for non-SWAp unpooled loans, prove to be excessive. Resource can be augmented in phases and be contingent on the Bank improving its portfolio performance and adopting a strategically focussed approach to addressing health sector problems.

• Portfolio Improvement: It is recommended that the Bank rigorously implement portfolio improvement measures, including those bearing on quality-at-entry, quality-atimplementation, and quality-at-exit67 in close collaboration with the RMCs to encourage them to play a more active role in the project cycle. Priority should be given to ESW in order to build the knowledge and evidence base for improving the relevance and strategic positioning of Bank interventions. This will enable the Bank to develop more appropriate country strategies and lending programmes.

• Results Orientation in Project Management: Working in close collaboration with other development partners, the Bank should develop a more effective set of tools that can track and demonstrate the project and programme impacts not only on the final outcome but also on intermediate outcomes. This implies that the Bank needs to take actions to strengthen the health information systems in the RMCs in order to enable them to set up adequate monitoring and evaluation systems for collecting reliable information including on intermediate outcomes. The staff incentive structure must incorporate monitoring and supervision activities as a factor in staff performance evaluation. The managers’ performance should include the consistency, the quality of self-evaluation and the actual performance which should benefit from a peer-review

• Efficiency in Staff Management: The Bank should provide administrative support to ,technical staff and consider increasing current staffing profiles and levels of both professional and administrative personnel. Technical staff should receive sustained training in project management techniques and methods including logical framework and results-based management techniques. It is also of paramount importance to the Bank to make sure that managers have the appropriate skills, capacity and incentives to manage for sustainable results on the ground instead of tactical behaviour responding to in-house requests and avoiding risk taking for innovative actions. Sufficient time should be allowed for peer review and this activity needs to be factored in the staff’s work programme and performance evaluation.