Travel time along a 13 km stretch of Nairobi’s Outer Ring Road has been eased considerably, with the opening of the two-lane dual carriageway in early 2019. This road improvement was thanks to an AfDB-funded project that was launched in 2015. As a bonus, the intervention led to reduced road fatalities, encouraged the establishment of new businesses, and restored security for the people who use the road. These outcomes come as no surprise given that the project was found to have had good “Quality at Entry” by a recent IDEV evaluation. This case study will argue that in particular evaluability and implementation readiness are the key quality at entry factors that can lead to a greater likelihood of project success.
Introduction
Quality at entry is defined as the state of preparedness that makes a project likely to succeed. When it is being designed, a project should fulfill certain conditions before it can be started. These conditions prepare the ground for the project to be executed as smoothly as possible. Most of all, they set the stage for the project to be able to deliver on its planned development outcomes.
Best practices in the development arena agree on four key dimensions (Figure 1) to assess quality when a project is being designed, to ensure that it meets good quality at entry standards. They are Evaluability, Implementation Readiness, Economic Analysis, and Risk Analysis.
Figure 1: Four key dimensions to assess project quality at entry
Source: Independent Evaluation of the Quality at Entry of African Development Bank Group Operations (2013-2017), October 2018
Of these four dimensions, Evaluability and Implementation Readiness were found to be key factors in predicting the performance of public sector investment projects by an IDEV Evaluation of the Quality at Entry of Operations funded by the AfDB.
Evaluability looks at the logical framework of an intervention or project. It reflects the extent to which: (i) the intervention logic is clear, responding to both the development problem and country context; (ii) the design of the intervention is supported by evidence; and (iii) the outcomes of the intervention are clear and measurable.
Implementation readiness, on the other hand, assesses the extent to which different implementation requirements for a project have been finalized, issues that could otherwise contribute to start-up delays for a project (see Figure 2).
Figure 2: Elements of good implementation readiness at project design
Source: IDEV Evaluation of Independent Evaluation of Quality Assurance across the Project Cycle of the African Development Bank Group (2012-2017), October 2018
The evaluation proved that by assessing two dimensions of a project during the design stage – Evaluability and Implementation Readiness – it is possible to predict if the project is likely to successfully deliver on its expected Development Outcomes.
A case in point is the AfDB funded Nairobi Outer Ring Road Improvement Project, which we will use to illustrate this evaluation’s findings.
The Nairobi Outer Ring Road Improvement Project
By 2013, traffic congestion on the Outer Ring Road in Kenya’s capital Nairobi was a nightmare for residents. Traveling along this road, especially during peak hours, was not only a great agony for commuters who spent long hours in traffic jams, but it led to the loss of revenue for businesses that plied the road as commuters rarely got out of the traffic to shop. At the time, a 13km stretch of single carriageway crossed a densely populated area of Nairobi, linking Nairobi city to various economic activity centers, such as the industrial zone, and providing secondary access to the Jomo Kenyatta International Airport, as well as to major highways across the country.
To relieve congestion on this highway, the government of Kenya approached the AfDB for a loan to expand the single carriageway into a 2-lane dual highway. Thus, the Nairobi Outer Ring Road Project was born.
According to the project design document, the Project Appraisal Report, the development objective was to “enhance economic efficiency through improved mobility and accessibility to businesses, thereby supporting economic and social development of the Nairobi city and the country at large”.
The expected outcomes were:
- Improved mobility and accessibility of the city of Nairobi and reduced congestion on Outer Ring Road;
- Local jobs created (youth from the nearby informal settlements would receive skills training under the project);
- Improved economic and social welfare of people living along the road corridor;
- Improved air quality to travelers and residents along the road.
The project was expected to cost about USD 133.7 million, with the AfDB covering 90% of the funding, while the government of Kenya would contribute 10%. The project was approved in June 2013 and projected to end in September 2018. The president of the Republic of Kenya, Uhuru Kenyatta, officially launched the road improvement project on 22nd January 2015. Four years later, the road corridor was almost fully completed and opened to traffic. [The project was extended to December 2019 to cater for added works that were not in the initial project design].
Evaluation’s Assessment of the Project’s Quality at Entry
From the outset, the Outer Ring Road Improvement Project was designed to be a success. It was found to have one of the highest ‘quality at entry’ ratings among the AfDB projects that IDEV evaluated for its evaluation of quality at entry. In particular, the Project Appraisal Report got a high rating on the two key dimensions of quality at entry, namely, evaluability and implementation readiness.
Evaluability
The project’s evaluability was assessed as highly satisfactory by the evaluation. The development problem was very clear: “Traffic congestion in and around the city, which resulted in economic uncertainty and mobility problems for the people who ply the road”. The project design responded to the identified problem, along with the main factors contributing to it, which were backed by evidence.
Figure 3. Causality chain in the Project’s Results-Based Logical Framework
Source: Reconstructed from the Nairobi Outer Ring Road Improvement Project Appraisal Report, October 2013
There was a clear linkage between the project’s outputs and outcomes in the project Results-Based Logical Framework. The causal linkage - from activities to outputs to outcomes, then to impacts - had been well demonstrated (Figure 3).
Furthermore, the indicators and targets for the outputs and outcomes were realistic and achievable during the lifetime of the project. (See Table 1)
Table 1. A Selection of Outcome indicators specified in the Project’s Results-Based Logical Framework
|
Outcome
|
Indicator |
Baseline value (value at project design) |
Most recent value |
End Target (expected value at project completion) |
Assessment |
|
1: Congestion on Outer Ring Road
|
volume/capacity ratio |
0.8 (2013)
|
0.83 (2020) |
0.4 (2018)
|
Unlikely to be achieved due to unanticipated increase in traffic levels and localized congestion in some sections |
|
2: Local jobs created
|
Cumulative number of persons employed on the project |
0.0 (2013) |
2100 (2018) |
2800 (2019)
|
Likely to be achieved |
Source: Extracted from AfDB Implementation Progress and Results Report, September 2019
Implementation Readiness
Figure 1 shows the elements of good implementation readiness at the design phase of any project. The cost estimate (in the Project Appraisal Report) of the Outer Ring Road project was based on up-to-date feasibility and detailed design studies, and a detailed work plan was established for the first 12 months of implementation. The project had a well-established implementation agency - the Kenya Urban Road Authority (KURA). KURA’s Financial Manual was found to be well suited to guide the financial management and disbursements, as well as the procurements for the project. Furthermore, KURA’s financial management capacity was assessed as adequate for the project.
At the design phase, managing environmental and social risks for the project was an important issue that had to be addressed. According to the Bank’s Environmental and Social Impact assessment study, the project was “likely to cause significant environmental and social impacts”. It was crucial to have a management plan for the likely adverse social and environmental impacts, including the involuntary resettlement of more than 200 people. The project, therefore, included a Full Resettlement Action Plan.
Overall, the evaluation rated the implementation readiness of the project design as highly satisfactory.
Development Outcomes Registered so far
According to the September 2019 Implementation Progress and Results Report, the project was set to deliver on its development outcomes as planned. The progress of the civil works was on course –87% complete as of September 2019.
Outcome 1 of the project was the reduction of the congestion on the Outer Ring Road. The indicator for this outcome was the volume to capacity ratio[1]. At project design (in 2013), the volume to capacity ratio along this road was 0.8. The target for this indicator was to bring the volume to capacity ratio down to 0.4 at the end of the project. By September 2019, a large section of the roadway had been opened up for public traffic use and the congestion relief was quite apparent. Recent traffic studies (2020) have shown that there has been an unanticipated increase in traffic levels along the project road. The current volume capacity ratio of the road is 0.83. Nevertheless, reduced travel times have been recorded. At project design, the travel time was between 2 and 3 hours; currently, this has been bought down to between 15 and 30 minutes. The increased traffic levels have been due to the increase in developments within the project corridor and the city. These developments include new shopping complexes (Moutain, Shujaa Malls) and residential estates (Greenspan, Komarock Heights) amongst others, which are among the impacts of the road improvement project.
Outcome 2 of the project was ‘Local jobs created’. By September 2019, a total of 2,100 people had been employed on the project, which was not far off the target of 2,800 set for the indicator. (See Table 1).
Lessons
It was important to carry out the feasibility and technical studies that informed the design of the Nairobi Outer Ring Road project. The studies provided evidence about the gravity of the development problem to the country, as well as the rationale for the project design. Most importantly, the studies helped to get the buy-in of the relevant authorities in the country.
While the expected target volume/capacity ratio (of 0.4) for the Outcome 1 indicator of the project has been deemed unlikely to be achieved, it is important to appreciate that the projected impacts (namely reduced travel times) are being achieved nonetheless. This shows that indicator targets may not tell the full story, because the project’s context can change over time. Thus, the non-achievement of set indicator targets does not imply that a project is not successful.
The existence of the implementing agency, KURA, was a huge plus for the project. For similar investment projects in other countries, the Bank has had to facilitate the setting up of a project implementation unit. This usually introduces bottlenecks such as delays in project start-up and more red tape in the running of the project.
Conclusion
The importance of establishing good quality at entry for development projects during the design phase cannot be over-emphasized. A project team and the quality assurance reviewers, or quality control measures of the project, must ensure that all the elements of good quality at entry are well articulated in the project design document. As mentioned at the start of this article, the four dimensions of good quality at entry are evaluability, implementation readiness, economic analysis, and risk analysis. However, of the four, evaluability and implementation readiness are the key ones to establish for a greater likelihood of project success. The Outer Ring Road Project showed that addressing these two dimensions at project design set the stage for the project to be executed as smoothly as possible. It also laid the groundwork for the project to deliver on its planned development outcomes and expected impact.
References:
- Independent Evaluation of the Quality at Entry of African Development Bank Group Operations (2013-2017)
- Independent Evaluation of the Quality of Supervision and Exit of African Development Bank Group Operations (2012-2017)
- Independent Evaluation of Quality Assurance across the Project Cycle of the African Development Bank Group (2012-2017)
- AfDB Kenya Nairobi Outer Ring Road Improvement - Project Appraisal Report
- AfDB Implementation Progress and Results Report, September 2019
[1] Volume to capacity ratio (V/C) is an index used to assess traffic status in cities. V is the total number of vehicles passing a point in one hour and C is the maximum number of cars that can pass a certain point during moderate traffic. As the V/C ratio approaches 1.0 and above, traffic delays and queues become longer.