Description
A Capital Improvement Plan (Program), or CIP, is a short-range plan, usually four to ten years, which identifies capital projects and equipment purchases, provides a planning schedule and identifies options for financing the plan. Essentially, the plan provides a link between a municipality, school district, parks and recreation department and/or other local government entity and a comprehensive and strategic plans and the entity's annual budget.
QUANTIFYING RISKS IN THE CAPITAL IMPROVEMENT PLAN: A Federal Tool for Local Success
Jason Gillespie – HDR Engineering, Inc. of the Carolinas
ABSTRACT
Long-term planning for a water or wastewater utility demands development of a Capital Improvement Plan (CIP) in order to tie identified projects or capital purchases to annual budgets. Too often however it seems that even when executed with in-house resources, projects are unable to be delivered within the timeframe and budget that has been approved. Is this experienced as a result of poor cost estimates, poor project managers, or poor budgeting decisions as is usually identified? Although any or all of the above issues may play a part in falling short of the goals for an individual project, the overarching issue of identifying risk in the CIP as a program should be addressed. Risks are inherent in the CIP in that there are uncertainties and unknowns in the planning process and as a result of the inter-relationships among project resources. These risks can be identified as one of two types: ? ? Baseline Risks which include price, quantity and schedule elements that will deviate from the estimate. Event Risks which are risks of internal or external events that force a project team to work outside of the basis of the estimate to meet the project scope. Event risks include both threats (increased costs or delays) and opportunities (reduced costs or time savings).
How well these risks are quantified and managed at the program level will directly translate to a more reliable and defensible CIP. This paper will present a case study in how risk is currently being planned for and managed in development of the federal government’s programmatic cost estimate for the Greater New Orleans Hurricane and Storm Damage Risk Reduction System. The US Army Corps of Engineers methods, outcomes, and applicability to the local CIP planning process will also be discussed.
KEYWORDS
Capital Improvement Plan, Risk Analysis, Risk Management, Cost Estimating
INTRODUCTION
Public projects are often highly publicized and criticized due to schedule and cost overruns. There is often mistrust in the ability of public agencies to manage and estimate the costs of these projects. As consultants and leaders of public agencies it is our responsibility to find new ways to adequately forecast total costs and schedule for public projects. More comprehensive, statistical approaches are being employed in order to meet this challenge.
METHODOLOGY
The US Army Corps of Engineers Greater New Orleans Hurricane and Storm Damage Risk Reduction System (GNOHSDRRS) Programmatic Schedule and Cost Estimate QUANTIFYING RISKS IN THE CAPITAL IMPROVEMENT PLAN: A Federal Tool for Local Success
According to the Office of the Federal Coordinator for Gulf Coast Rebuilding, hurricanes Katrina and Rita affected 90,000 square miles—an area the size of Great Britain. During Hurricane Katrina, over 80 percent of New Orleans flooded—an area seven times the size of Manhattan. More than 1.5 million people were directly affected and more than 800,000 citizens were forced to live outside of their homes—the largest displacement of people since the great Dust Bowl migrations of the 1930s. Hurricanes Katrina and Rita were two of the most costly national disasters to occur in the United States. The total losses of physical capital, including housing, consumer durable goods, losses in the energy sector, other private sector, and in the public sector, were estimated to be $130 billion for both hurricanes as estimated by Risk Management Solutions for the Congressional Budget Office Testimony in October 2005. Restoring the confidence of those who live in south Louisiana within the limits of the GNOHSDRRS is a major goal of the U.S. Army Corps of Engineers as it implements a multi-year effort to rebuild and restore levees, floodwalls and pump stations. In the year after Katrina, USACE compressed the work of many years by coming up with a plan to improve New Orleans hurricane protection. Quantifying the scope of work to be completed, time table and budget was key to managing the overall program. The Corps chose to develop a risk based program schedule and programmatic cost estimate to deliver to Congress. Why Risk Analysis Economic forecasts traditionally take the form of a single “expected outcome” supplemented with alternative scenarios. The limitation of a forecast with a single expected outcome is clear -- while it may provide the single best statistical estimate, it offers no information about the range of other possible outcomes and their associated probabilities. The problem becomes acute when uncertainty surrounding the forecast’s underlying assumptions is material. A common approach is to create “high case” and “low case” scenarios to bracket the central estimate. This scenario approach can exacerbate the problem of dealing with risk because it gives no indication of likelihood associated with the alternative outcomes. The commonly reported “high case” may assume that most underlying assumptions deviate in the same direction from their expected value, and likewise for the “low case.” In reality, the likelihood that all underlying factors shift in the same direction simultaneously is just as remote as that of everything turning out as expected. Another common approach to providing added perspective on reality is “sensitivity analysis.” Key forecast assumptions are varied one at a time in order to assess their relative impact on the expected outcome. A problem here is that the assumptions are often varied by arbitrary amounts. A more serious concern with this approach is that, in the real world, assumptions do not veer from actual outcomes one at a time. It is the impact of simultaneous differences between assumptions and actual outcomes that is needed to provide a realistic perspective on the risk of a forecast. Risk analysis provides a way around the problems outlined above. It helps avoid the lack of perspective in “high” and “low” cases by measuring the probability or “odds” that an outcome will actually materialize. This is accomplished by attaching ranges (probability distributions) to the forecasts of each input variable. The approach allows all inputs to be varied simultaneously within their distributions, thus avoiding the problems inherent in conventional sensitivity analysis. The approach also recognizes interrelationships between variables and their associated probability distributions. The risk analysis process used involves four major steps: QUANTIFYING RISKS IN THE CAPITAL IMPROVEMENT PLAN: A Federal Tool for Local Success
Step 1-Define the structure and logic of the forecasting problem. A “structure and logic model” depicts the variables and cause and effect relationships that underpin the forecasting problem at-hand. Although the structure and logic model is written down mathematically to facilitate analysis, it is also depicted diagrammatically in order to permit stakeholder scrutiny and modification in Step 3 of the process. Step 2-Assign estimates and ranges (probability distributions) to each variable and model coefficient in the forecasting structure and logic. Each variable is assigned a central estimate and a range (probability distribution). Special data sheets are used to record the estimates based on median, upper and lower values. The greater the uncertainty associated with a forecast variable the wider the range of possible values the variable can take on. Probability ranges are established on the basis of both statistical analysis and subjective probability. Probability ranges need not be normal or symmetrical; that is, there is no need to assume the bell shaped normal probability curve. The bell curve assumes an equal likelihood of being too low and being too high in forecasting a particular value. It might well be, for example, that if a projected growth rate deviates from expectations, circumstances are such that it is more likely to be higher than the median expected outcome than lower. Step 3-Engage experts and stakeholders in an assessment of the model and risk assumptions in workshops. Step 3 involves the formation of an expert panel and the use of facilitation techniques to elicit, from the panel, risk and probability beliefs about: 1. 2. The structure of the forecasting framework; and Uncertainty attached to each variable and forecasting coefficient within the framework.
In (1), experts are invited to add variables and hypothesized causal relationships that may be material, yet missing from the model. In (2), panelists are engaged in a discursive protocol during which the frequentist-based central estimates and ranges are modified according to subjective expert beliefs. Step 4-Issue risk analysis outcomes. The final probability distributions are formulated by the risk analyst and represent a combination of “frequentist” and subjective probability information drawn from Step 3. These are combined using a simulation technique (Monte Carlo analysis) that allows each variable and forecasting coefficient to vary simultaneously according to its associated probability distribution. Defining Risk Impacts Risk impacting projects can be broadly classified into two categories: baseline risks and event risks. Baseline Risks Baseline Risks include price, quantity and schedule elements that will deviate from the estimate. Examples include variations in unit prices, deviations in material quantities, and the range duration of activities. Baseline uncertainty is typically defined by lower and upper parameters of a distribution, such as the 10% and 90% percentiles. These baseline risk distributions are applied in the model by converting each single baseline estimate into a variable that is defined by its distribution. During simulations, the baseline estimate varies according to the distribution, drawing a probabilistically-determined estimate each time. Event Risks Event risks are risks of internal or external events that force the project team to work outside of the basis of the estimate to meet the project scope. Event risks include both threats (increased QUANTIFYING RISKS IN THE CAPITAL IMPROVEMENT PLAN: A Federal Tool for Local Success
costs or delays) and opportunities (reduced costs or time savings). Examples include changes in key staff, environmental compliance delays, contract modifications, or long lead times for materials. Opportunities include value engineering cost savings. Event risks have a probability or likelihood of occurring and can impact project costs, project schedule, or both. The cost impacts of event risks are elicited during the workshop sessions, and organized in a Risk Register. Workshop participants are asked to identify and discuss potential event risks and assign a range of possible values (most likely, low, high) to populate that distribution. The schedule impacts of event and scope risks are elicited in a similar way.
RESULTS
The outcomes of the risk analysis include, for each scenario: ? ? ? A set of probable cost and schedule outcomes arrayed against their respective probability of occurrence (i.e., a full probability distribution for total project costs, and expected completion date); The identification of key risk factors and their estimated impact on project costs and schedule; and The identification of mitigation strategies and their estimated impact on project costs and schedule;
Risk analysis for project construction costs is often compared to the “contingency” line item in an engineer’s cost estimate sheet. This risk analysis approach is a means to not only refine the contingency estimate at any stage of the project cycle, but also to better understand, quantify, and mitigate the different risk items entering, often implicitly, the estimation of the contingency amount.
DISCUSSION
A risk assessment based approach provides a planning tool for determining the probability of project schedules and costs. The process ultimately results in cost and schedule estimates related to the impacts of various risk factors. It also helps decision makers in developing mitigation strategies and identifying opportunities for cost control and reduction. The risk assessment method adds value to the overall CIP process by: ? ? ? ? Encouraging proactive planning Providing the collaborative environment to develop mitigation strategies Building confidence and credibility in the project plans and estimates Providing better cost and schedule forecasts for planning, budgeting and bonding.
REFERENCES
The Evans-Graves Team, 2007, Cost and Schedule Risk Analysis Report for the Hurricane Protection System prepared for Task Force Hope, US Army Corps of Engineers-Mississippi Valley Division.
QUANTIFYING RISKS IN THE CAPITAL IMPROVEMENT PLAN: A Federal Tool for Local Success
doc_201911162.pdf
A Capital Improvement Plan (Program), or CIP, is a short-range plan, usually four to ten years, which identifies capital projects and equipment purchases, provides a planning schedule and identifies options for financing the plan. Essentially, the plan provides a link between a municipality, school district, parks and recreation department and/or other local government entity and a comprehensive and strategic plans and the entity's annual budget.
QUANTIFYING RISKS IN THE CAPITAL IMPROVEMENT PLAN: A Federal Tool for Local Success
Jason Gillespie – HDR Engineering, Inc. of the Carolinas
ABSTRACT
Long-term planning for a water or wastewater utility demands development of a Capital Improvement Plan (CIP) in order to tie identified projects or capital purchases to annual budgets. Too often however it seems that even when executed with in-house resources, projects are unable to be delivered within the timeframe and budget that has been approved. Is this experienced as a result of poor cost estimates, poor project managers, or poor budgeting decisions as is usually identified? Although any or all of the above issues may play a part in falling short of the goals for an individual project, the overarching issue of identifying risk in the CIP as a program should be addressed. Risks are inherent in the CIP in that there are uncertainties and unknowns in the planning process and as a result of the inter-relationships among project resources. These risks can be identified as one of two types: ? ? Baseline Risks which include price, quantity and schedule elements that will deviate from the estimate. Event Risks which are risks of internal or external events that force a project team to work outside of the basis of the estimate to meet the project scope. Event risks include both threats (increased costs or delays) and opportunities (reduced costs or time savings).
How well these risks are quantified and managed at the program level will directly translate to a more reliable and defensible CIP. This paper will present a case study in how risk is currently being planned for and managed in development of the federal government’s programmatic cost estimate for the Greater New Orleans Hurricane and Storm Damage Risk Reduction System. The US Army Corps of Engineers methods, outcomes, and applicability to the local CIP planning process will also be discussed.
KEYWORDS
Capital Improvement Plan, Risk Analysis, Risk Management, Cost Estimating
INTRODUCTION
Public projects are often highly publicized and criticized due to schedule and cost overruns. There is often mistrust in the ability of public agencies to manage and estimate the costs of these projects. As consultants and leaders of public agencies it is our responsibility to find new ways to adequately forecast total costs and schedule for public projects. More comprehensive, statistical approaches are being employed in order to meet this challenge.
METHODOLOGY
The US Army Corps of Engineers Greater New Orleans Hurricane and Storm Damage Risk Reduction System (GNOHSDRRS) Programmatic Schedule and Cost Estimate QUANTIFYING RISKS IN THE CAPITAL IMPROVEMENT PLAN: A Federal Tool for Local Success
According to the Office of the Federal Coordinator for Gulf Coast Rebuilding, hurricanes Katrina and Rita affected 90,000 square miles—an area the size of Great Britain. During Hurricane Katrina, over 80 percent of New Orleans flooded—an area seven times the size of Manhattan. More than 1.5 million people were directly affected and more than 800,000 citizens were forced to live outside of their homes—the largest displacement of people since the great Dust Bowl migrations of the 1930s. Hurricanes Katrina and Rita were two of the most costly national disasters to occur in the United States. The total losses of physical capital, including housing, consumer durable goods, losses in the energy sector, other private sector, and in the public sector, were estimated to be $130 billion for both hurricanes as estimated by Risk Management Solutions for the Congressional Budget Office Testimony in October 2005. Restoring the confidence of those who live in south Louisiana within the limits of the GNOHSDRRS is a major goal of the U.S. Army Corps of Engineers as it implements a multi-year effort to rebuild and restore levees, floodwalls and pump stations. In the year after Katrina, USACE compressed the work of many years by coming up with a plan to improve New Orleans hurricane protection. Quantifying the scope of work to be completed, time table and budget was key to managing the overall program. The Corps chose to develop a risk based program schedule and programmatic cost estimate to deliver to Congress. Why Risk Analysis Economic forecasts traditionally take the form of a single “expected outcome” supplemented with alternative scenarios. The limitation of a forecast with a single expected outcome is clear -- while it may provide the single best statistical estimate, it offers no information about the range of other possible outcomes and their associated probabilities. The problem becomes acute when uncertainty surrounding the forecast’s underlying assumptions is material. A common approach is to create “high case” and “low case” scenarios to bracket the central estimate. This scenario approach can exacerbate the problem of dealing with risk because it gives no indication of likelihood associated with the alternative outcomes. The commonly reported “high case” may assume that most underlying assumptions deviate in the same direction from their expected value, and likewise for the “low case.” In reality, the likelihood that all underlying factors shift in the same direction simultaneously is just as remote as that of everything turning out as expected. Another common approach to providing added perspective on reality is “sensitivity analysis.” Key forecast assumptions are varied one at a time in order to assess their relative impact on the expected outcome. A problem here is that the assumptions are often varied by arbitrary amounts. A more serious concern with this approach is that, in the real world, assumptions do not veer from actual outcomes one at a time. It is the impact of simultaneous differences between assumptions and actual outcomes that is needed to provide a realistic perspective on the risk of a forecast. Risk analysis provides a way around the problems outlined above. It helps avoid the lack of perspective in “high” and “low” cases by measuring the probability or “odds” that an outcome will actually materialize. This is accomplished by attaching ranges (probability distributions) to the forecasts of each input variable. The approach allows all inputs to be varied simultaneously within their distributions, thus avoiding the problems inherent in conventional sensitivity analysis. The approach also recognizes interrelationships between variables and their associated probability distributions. The risk analysis process used involves four major steps: QUANTIFYING RISKS IN THE CAPITAL IMPROVEMENT PLAN: A Federal Tool for Local Success
Step 1-Define the structure and logic of the forecasting problem. A “structure and logic model” depicts the variables and cause and effect relationships that underpin the forecasting problem at-hand. Although the structure and logic model is written down mathematically to facilitate analysis, it is also depicted diagrammatically in order to permit stakeholder scrutiny and modification in Step 3 of the process. Step 2-Assign estimates and ranges (probability distributions) to each variable and model coefficient in the forecasting structure and logic. Each variable is assigned a central estimate and a range (probability distribution). Special data sheets are used to record the estimates based on median, upper and lower values. The greater the uncertainty associated with a forecast variable the wider the range of possible values the variable can take on. Probability ranges are established on the basis of both statistical analysis and subjective probability. Probability ranges need not be normal or symmetrical; that is, there is no need to assume the bell shaped normal probability curve. The bell curve assumes an equal likelihood of being too low and being too high in forecasting a particular value. It might well be, for example, that if a projected growth rate deviates from expectations, circumstances are such that it is more likely to be higher than the median expected outcome than lower. Step 3-Engage experts and stakeholders in an assessment of the model and risk assumptions in workshops. Step 3 involves the formation of an expert panel and the use of facilitation techniques to elicit, from the panel, risk and probability beliefs about: 1. 2. The structure of the forecasting framework; and Uncertainty attached to each variable and forecasting coefficient within the framework.
In (1), experts are invited to add variables and hypothesized causal relationships that may be material, yet missing from the model. In (2), panelists are engaged in a discursive protocol during which the frequentist-based central estimates and ranges are modified according to subjective expert beliefs. Step 4-Issue risk analysis outcomes. The final probability distributions are formulated by the risk analyst and represent a combination of “frequentist” and subjective probability information drawn from Step 3. These are combined using a simulation technique (Monte Carlo analysis) that allows each variable and forecasting coefficient to vary simultaneously according to its associated probability distribution. Defining Risk Impacts Risk impacting projects can be broadly classified into two categories: baseline risks and event risks. Baseline Risks Baseline Risks include price, quantity and schedule elements that will deviate from the estimate. Examples include variations in unit prices, deviations in material quantities, and the range duration of activities. Baseline uncertainty is typically defined by lower and upper parameters of a distribution, such as the 10% and 90% percentiles. These baseline risk distributions are applied in the model by converting each single baseline estimate into a variable that is defined by its distribution. During simulations, the baseline estimate varies according to the distribution, drawing a probabilistically-determined estimate each time. Event Risks Event risks are risks of internal or external events that force the project team to work outside of the basis of the estimate to meet the project scope. Event risks include both threats (increased QUANTIFYING RISKS IN THE CAPITAL IMPROVEMENT PLAN: A Federal Tool for Local Success
costs or delays) and opportunities (reduced costs or time savings). Examples include changes in key staff, environmental compliance delays, contract modifications, or long lead times for materials. Opportunities include value engineering cost savings. Event risks have a probability or likelihood of occurring and can impact project costs, project schedule, or both. The cost impacts of event risks are elicited during the workshop sessions, and organized in a Risk Register. Workshop participants are asked to identify and discuss potential event risks and assign a range of possible values (most likely, low, high) to populate that distribution. The schedule impacts of event and scope risks are elicited in a similar way.
RESULTS
The outcomes of the risk analysis include, for each scenario: ? ? ? A set of probable cost and schedule outcomes arrayed against their respective probability of occurrence (i.e., a full probability distribution for total project costs, and expected completion date); The identification of key risk factors and their estimated impact on project costs and schedule; and The identification of mitigation strategies and their estimated impact on project costs and schedule;
Risk analysis for project construction costs is often compared to the “contingency” line item in an engineer’s cost estimate sheet. This risk analysis approach is a means to not only refine the contingency estimate at any stage of the project cycle, but also to better understand, quantify, and mitigate the different risk items entering, often implicitly, the estimation of the contingency amount.
DISCUSSION
A risk assessment based approach provides a planning tool for determining the probability of project schedules and costs. The process ultimately results in cost and schedule estimates related to the impacts of various risk factors. It also helps decision makers in developing mitigation strategies and identifying opportunities for cost control and reduction. The risk assessment method adds value to the overall CIP process by: ? ? ? ? Encouraging proactive planning Providing the collaborative environment to develop mitigation strategies Building confidence and credibility in the project plans and estimates Providing better cost and schedule forecasts for planning, budgeting and bonding.
REFERENCES
The Evans-Graves Team, 2007, Cost and Schedule Risk Analysis Report for the Hurricane Protection System prepared for Task Force Hope, US Army Corps of Engineers-Mississippi Valley Division.
QUANTIFYING RISKS IN THE CAPITAL IMPROVEMENT PLAN: A Federal Tool for Local Success
doc_201911162.pdf