Description
A budget is a quantitative expression of a plan for a defined period of time. It may include planned sales volumes and revenues, resource quantities, costs and expenses, assets, liabilities and cash flows. It expresses strategic plans of business units, organizations, activities or events in measurable terms
E. K.., January 2005
Quantifying the Effects of Budget Management
on Project Cost and Success
Ed Kujawski
Engineering Division
Lawrence Berkeley National Laboratory
E-mail: [email protected]
Tel: 510.486.6932
INCOSE Meeting – San Francisco Bay Area Chapter
January 11, 2005
E. K.., January 2005
Outline
× Based on E. Kujawski, M.L. Alvaro, W.R. Edwards, “Incorporating Psychological
Influences in Probabilistic Cost Analysis”, Systems Engineering Vol. 7, No. 3, 2004.
4 The cost overrun problem and its causes
– (Organizational considerations), human behavior, modeling
4 A modified PCA
– Level of analysis
– Assessment of cost elements
– Distribution Functions
– MAIMS principle
– Two-level correlation model
4 Analysis of a sample design/engineering project
4 Budget allocation, contingency management, and project cost
4 Summary of key concepts
– Comparison with other approaches
4 Future directions
E. K.., January 2005
The cost-overrun problem
× This observation is very insightful and still applicable today.
4 Common threads among the various “top 10” lists
– Institutional and organizational culture
• Procurement process, management pressure, poor project definition
– Real Vs. idealized human behavior
• Psychology is relevant to economics,decision-making,management,...
eThe “100% rational” person is a theoretical model that differs from reality.
– Inadequate analyses - Today’s typical PCA
• Ad-hoc data elicitation, improper distributions, omitted and/or limited
dependencies, omitted high risk events & decision points
eShift from deterministic to probabilistic approach is NOT silver bullet !
• Monte Carlo simulation is only a mathematical tool: GIGO.
– Poor management practices
• Lack of appreciation of probabilistic concepts and psychological influences in
budget allocation and control of management reserve
® Projects that come-in under cost do not necessarily deserve kudos.
– They may have carried excessively safe budgets.
"Their judgment was based more on wishful thinking
than on sound calculation of probabilities.”
Thucydides, 431 B.C.E.
E. K.., January 2005
Current project reality leads to cost
overruns
Our approach models these causes and effects to obtain
realistic cost estimates and enhance project success.
Win project
H
Some leads want
safe estimates
Low project
cost
estimates
P
Today's typical
PCA
Project cost
overruns
P
Inadequate
project
management
H
Achieve technical
performance
O
Management
wants to meet
schedule
H
Human
behavior
O
Organization/
Politics*
Legend
H
Optimism about
technolgy
Conflict
O
Management
pressure for low
estimates
Conflict
Conflict
P
Practices
* beyond scope
E. K.., January 2005
Psychology can teach us much about
cost overruns
× Overconfidence
– R&D folks are intrinsically optimistic about new technologies.
– "For heaven's sake, Spread Those Fractiles! Be honest with yourselves! Admit
what you don't know!" Alpert and Raiffa, 1982
× Negative human behavior – MAIMS Principle
– "Money Allocated Is Money Spent.“ C. Gordon, 1997
Task underruns are rarely available to protect against tasks overruns.
Task overruns are passed on to the total project cost.
× Mistakes of reason
– “Too many details tend to cloud the big picture.”
Total project cost is not simply the sum of the individual cost elements.
Project characteristics and risks are likely to affect multiple elements.
– “Implicitly trusting the most readily available information or anchoring too much
on convenient facts.” Russo and Schoemaker, 1990 - Decision trap # 5
Realistic cost analysis requires a systems engineering approach.
A credible cost analysis needs to integrate psychological findings
with mathematically valid models and sound management techniques.
E. K.., January 2005
Insight into structuring PCA
Consider n cost elements with uncertainty at WBS level-i
- Total project cost random variable
- Expect value
- Variance
e Consider Independent cost elements
- Fictitious reduction of uncertainty
Central limit theorem applies
- C
T
is a Gaussian normal distribution!
) ( ) (
?
=
n
j
j T C E C E
?
=
n
j
j T C C
1 ) , ( 1
)] ( * ) ( [ * ) , ( ) ( ) (
2 / 1
+ ? ? ?
+ =
? ??
?
j i
j i
j j j i
j i j T
C C Corr
C Var C Var C C Corr C Var C Var
n C E C Var T T / 1 ) ( / ) ( ?
Do not subdivide project cost into too many bite-size pieces!
Sum of 10 identical cost elements
?
?
E. K.., January 2005
Direct Fractile Assessment (DFA)
method
× Subjective assessment of cost elements
– DFA proven one of the most reliable and least bias-prone procedures for
eliciting uncertain quantities
· Experts provide 10
th
, 50
th
, and 90
th
percentiles
· Calibrate percentiles
– Default correction for optimism: 10
th
20
th
, 90
th
80
th
× Selection of realistic and flexible PDFs - Criteria
– C1. Fit 3 arbitrary percentiles
– C2. Finite lower range
– C3. Infinite upper range with reasonable behavior
– C4. Physically meaningful and easy to estimate parameters
The DFA method with calibration and
3-parameter Weibull can provide cost credibility
E. K.., January 2005
It’s the assessment!
× Values of the input percentiles
have a significant impact
× Important to select PDFs that fit
assessed percentiles
– Use Crystal Ball or @Risk
× Differences among fits where
expert opinion is unreliable
0%
20%
40%
60%
80%
100%
300 400 500 600 700 800 900
Cost , K$
C
o
s
t
p
r
o
b
a
b
i
l
i
t
y
o
f
s
u
c
c
e
s
s
W 10/50/90 W 20/50/80
B 10/50/90 B 20/50/80
Distribution
(fractiles*)
Mean
K$
SD^
K$
Weibull(10/50/90) 432 49
Weibull(20/50/80) 448 85
Beta(10/50/90) 432 45
Beta(20/50/80) 439 60
* 10/50/90: x
10
=382K$, x
50
=421K$, x
90
=499K$
20/50/80: x
20
=382K$, x
50
=421K$, x
80
=499K$.
E. K.., January 2005
MAIMS significantly impacts PCA cost
elements
Illustration- Cost element with 3-parameter Weibull distribution
Al l ocated
budget
X*, K$
Mean,
K$
Perc. of
mean
SD,
K$
Ideal 448 63 85
mean = 448 479 75 66
X50 = 422 463 72 73
X75 = 482 502 81 55
X85 = 521 535 87 48
4 Properties of MAIMS-Modified distributions
– Proper PDFs
– Minimum value: allocated budget, x*
– Modified Dirac delta function at x*
– Identical to original cost element for values > x*
¨ Not the same as Crystal Ball and @Risk truncated PDFs
MAIMS has a significant impact on PCA.
Impact increases with increased budget allocation.
E. K.., January 2005
A correlation model for dependencies at
the subsystem and system levels
4 There are multiple dependencies among cost elements
– Within a given subsystem due to technical complexity and common staff
– Among different subsystems due to common organizational and programmatic
considerations
¯ Consider cost elements C
m.j
- 1st and 2nd integers refer to WBS level 2 and level 3, respectively
» We model cost correlations based on Markowitz’s multi-factor model
C
m.j
= R
m.j
+ ?
m.j*
F
m
? ?
m.j
are constants; R
m.j
are independent random variables; F
m
are correlated random
variables; R
m.j
and F
n
are independent
? It can be shown Corr(C
m.j
, C
n.k
) = Corr(F
n
, F
m
)*?
m.j
*?
n.k
4 Given the lack of data, we make the following assumptions
- Simplified Two-Level Correlation Model (STLCM)
1. Corr(C
m.j
, C
m.k
) = ?
int
for cost elements in the same subsystem
2. Corr(C
m.j
, C
n.k
) = ?
ext
for cost elements in different subsystems
3. ?
int
> ?
ext
Important interrelationships in TLCM: system complexities,
staff, organizational and programmatic influences
E. K.., January 2005
Illustrative Analysis
Sample design/engineering project
WBS Cost Elements
Estimated Percentiles
K$
X10 X50 X90
1.0 Total project/system, CT
1.1 Project/system-level, C1
1.1.1 Project management, C1.1 382 421 499
1.1.2 Systems engineering, C1.2 220 232 257
1.1.3 Integration & test, C1.3 887 1,010 1,256
1.2 Subsystem X, C2
1.2.1 Mechanical components, C2.1 970 1088 1,323
1.2.2 Electrical components, C2.2 742 846 1,054
1.2.3 Integration & test, C2.3 596 724 980
1.3 Subsystem Y, C3
1.3.1 Software development, C3.1 1,069 1,282 1,708
1.3.2 Firmware, C3.2 634 743 961
1.3.3 Integration & test, C3.3 541 656 886
O Procedure
1. Establish CWBS
2. Assess cost elements
» Direct fractile assessment method
• 3 percentiles
• engineering judgment, experience,
& available data
3. Calibrate estimates
4. Fit estimates
» Three-parameter Weibull
5. Allocate budget to each cost element
6. Modify each cost element for MAIMS
7. Model correlation among cost elements
» Two-level correlation model
8. Perform Monte Carlo Simulation
9. Establish PoS
10. Determine total cost & contingency
» Modified PM approach
E. K.., January 2005
Budget allocation impacts project
cost and probability of success
4 Ideal Project
- “100% rational” team
- Each cost manager spends money only
as necessary to satisfy requirements
- Savings are available to support other
cost elements on an as-needed basis
· Actual costs may be less than
budgeted costs
4 Real Projects
- Human behavior and organizational
considerations
- MAIMS principle
- Budget and contingency management
are important confounding factors
- Effects increase with higher
allocated budgets and are
substantial
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
6,000 6,500 7,000 7,500 8,000 8,500 9,000 9,500
Cost, K$
MAIMS @ X50 MAIMS @ mean
MAIMS @ X75 Ideal
E. K.., January 2005
A tale of a project cost overrun
(1 of 2)
1. Agency X issues a RFP
- Requests cost at 50% CL
2. Contractor A prepares bid
© possesses limited sophistication;
but not cognizant of MAIMS
principle
- Develops CWBS
- Performs today’s typical PCA
• P50: 7,348 K$
• Min: 5,633 K$
3. Contractor A submits bid of
7,348 K$
? Confident he will succeed. Thinks
cost estimate has a 30% margin.
4. Contractor A is winner!
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
6,000 6,500 7,000 7,500 8,000 8,500 9,000 9,500
Cost, K$
MAIMS @X50 MAIMS @mean
MAIMS @X75 Ideal
E. K.., January 2005
A tale of a project cost overrun
(2 of 2)
5. The project starts & budgets are
allocated
- The practice is to baseline the Level-3
elements at mean values
• Baseline cost: 7,665 K$
® But project bid is 7,348 K$!
6. Much time is spent reallocating and
prorating budgets
- Budget cost elements at 50% CL
• Baseline cost: 7,002 K$
• Management reserve: ~ 5%
7. The outcome
® Everybody works very hard. But the
project runs out of budget and is
cancelled.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
6,000 6,500 7,000 7,500 8,000 8,500 9,000 9,500
Cost, K$
MAIMS @X50 MAIMS @mean
MAIMS @X75 Ideal
Epilogue
-Another project has succumbed to the MAIMS principle.
-Today’s typical PCA models a mythical project.
-Future RFPs, contracting agencies & contractors use proposed approach.
E. K.., January 2005
It’s NOT your classical contingency
anymore!
4 Cost contingency depends on desired probability of success and cost
management strategy
– MCC(PoS, PBC
1
,…,PBC
n
) = TEC(PoS, PBC
1
,…,PBC
n
) – PBC.
• MCC: Management Cost Contingency
• TEC: Total Estimated Cost
• PoS: Probability of Success
• PBC
i
: Baseline Budget for Cost element C
i
• PBC: sum over all cost elements.
» Management strategies and desired probabilities of success vary across business
categories
× Major differences with both deterministic practice and today’s typical PCA
» MCC is NOT a fixed percentage of PBC
» MCC incorporates MAIMS principle and depends on the management strategy
» Interactive and iterative process: system analysts, engineers, management
E. K.., January 2005
Realistic budget allocation, adequate contingency, and dynamic allocation
are critical to optimal cost and probability of success
Contingency, cost, & success
are NOT directly related
× High cost NEED NOT provide (1) high PoS or CL and/or (2) high contingency
× Low contingency DOES NOT necessarily equate to low cost
× High contingency DOES NOT necessarily equate to high cost and/or padding
5,500
6,000
6,500
7,000
7,500
8,000
8,500
9,000
9,500
10,000
10,500
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Confidence Level of Total Estimated Cost
T
o
t
a
l
E
s
t
i
m
a
t
e
d
C
o
s
t
,
K
$
0%
10%
20%
30%
40%
50%
60%
M
a
n
a
g
e
m
e
n
t
C
o
s
t
C
o
n
t
i
n
g
e
n
c
y
TEC MAIMS_@_mean TEC MAIMS_@_75 TEC MAIMS_@_50 TEC Ideal
MCC MAIMS_@_mean MCC MAIMS_@_75 MCC MAIMS_@_50
Different representation of data in previous figure
E. K.., January 2005
» Assessment of the cost elements
» Correlation effects
» Budget allocation
» Human behavior
» Organizational considerations
There are many confounding factors
to consider
These are important and confounding
factors that should be modeled
simultaneously in a PCA.
E. K.., January 2005
Our approach integrates many key concepts
Critical Chain
1
RACM
2
Today's Typical
PCA
Proposed Approach
Parameter Schedule Cost Cost Cost
Assessment of
uncertainty
- "Realistic" task
schedule = "Safe"
estimate/2
- Gaussian normal
PDFs
- Largely ad-hoc
- Extensive use of
triangular PDFs
- 3 percentiles using DFA
method
- 3-parameter Weibull
- Human
behavior
-Organizational
influences
- Parkinson' s law
- Safe estimates
- Multi-tasking
- MAIMS
- "Hidden"
incentives
"Ideal" project
& "100%
rational" person
- Calibrate cost elements
- Psychological findings
- MAIMS principle
Correlations Basic task
dependencies
None Limited, single
parameter model
Two-level correlation
model
Calculation
method
Deterministic, single-
point estimate
- Analytical/
statistical sum
Monte Carlo
simulation
Monte Carlo simulation
Project
management
- Project buffer
- Feeding buffers
- Project buffer: 25%
of original estimate
- Baseline budget
- Management
reserve
- Statistical cost
control
- Cost account
level and/or
management
reserve
- Baseline budget
- Management reserve
- Dynamic allocation
1
Goldratt's basic approach; numerous variations have been proposed
2
C. Gordon, Risk Analysis and Cost Management, Lockheed 1990's
E. K.., January 2005
Future directions
- Presented work focused on cost and macroscopic perspective
– it provides a framework for more accurate predictions
– it results in more realistic expectations
– benefits are likely to be significant
• more viable plans, better decisions, reduction in cost overruns.
× Much remains to be done
– integrate microscopic and macroscopic approaches
– simultaneously treat performance/cost/schedule
– quantitative calibration of data elicitation - single and multiple experts
Greatest challenge- implementation of systems thinking at the personnel,
organizational and institutional levels
- tool to dynamically adjust budget and modify negative behavior
- SE research to deal with psychological findings on human behavior and judgment
under uncertainty
Proposed approach is worth the additional effort!
doc_302300473.pdf
A budget is a quantitative expression of a plan for a defined period of time. It may include planned sales volumes and revenues, resource quantities, costs and expenses, assets, liabilities and cash flows. It expresses strategic plans of business units, organizations, activities or events in measurable terms
E. K.., January 2005
Quantifying the Effects of Budget Management
on Project Cost and Success
Ed Kujawski
Engineering Division
Lawrence Berkeley National Laboratory
E-mail: [email protected]
Tel: 510.486.6932
INCOSE Meeting – San Francisco Bay Area Chapter
January 11, 2005
E. K.., January 2005
Outline
× Based on E. Kujawski, M.L. Alvaro, W.R. Edwards, “Incorporating Psychological
Influences in Probabilistic Cost Analysis”, Systems Engineering Vol. 7, No. 3, 2004.
4 The cost overrun problem and its causes
– (Organizational considerations), human behavior, modeling
4 A modified PCA
– Level of analysis
– Assessment of cost elements
– Distribution Functions
– MAIMS principle
– Two-level correlation model
4 Analysis of a sample design/engineering project
4 Budget allocation, contingency management, and project cost
4 Summary of key concepts
– Comparison with other approaches
4 Future directions
E. K.., January 2005
The cost-overrun problem
× This observation is very insightful and still applicable today.
4 Common threads among the various “top 10” lists
– Institutional and organizational culture
• Procurement process, management pressure, poor project definition
– Real Vs. idealized human behavior
• Psychology is relevant to economics,decision-making,management,...
eThe “100% rational” person is a theoretical model that differs from reality.
– Inadequate analyses - Today’s typical PCA
• Ad-hoc data elicitation, improper distributions, omitted and/or limited
dependencies, omitted high risk events & decision points
eShift from deterministic to probabilistic approach is NOT silver bullet !
• Monte Carlo simulation is only a mathematical tool: GIGO.
– Poor management practices
• Lack of appreciation of probabilistic concepts and psychological influences in
budget allocation and control of management reserve
® Projects that come-in under cost do not necessarily deserve kudos.
– They may have carried excessively safe budgets.
"Their judgment was based more on wishful thinking
than on sound calculation of probabilities.”
Thucydides, 431 B.C.E.
E. K.., January 2005
Current project reality leads to cost
overruns
Our approach models these causes and effects to obtain
realistic cost estimates and enhance project success.
Win project
H
Some leads want
safe estimates
Low project
cost
estimates
P
Today's typical
PCA
Project cost
overruns
P
Inadequate
project
management
H
Achieve technical
performance
O
Management
wants to meet
schedule
H
Human
behavior
O
Organization/
Politics*
Legend
H
Optimism about
technolgy
Conflict
O
Management
pressure for low
estimates
Conflict
Conflict
P
Practices
* beyond scope
E. K.., January 2005
Psychology can teach us much about
cost overruns
× Overconfidence
– R&D folks are intrinsically optimistic about new technologies.
– "For heaven's sake, Spread Those Fractiles! Be honest with yourselves! Admit
what you don't know!" Alpert and Raiffa, 1982
× Negative human behavior – MAIMS Principle
– "Money Allocated Is Money Spent.“ C. Gordon, 1997
Task underruns are rarely available to protect against tasks overruns.
Task overruns are passed on to the total project cost.
× Mistakes of reason
– “Too many details tend to cloud the big picture.”
Total project cost is not simply the sum of the individual cost elements.
Project characteristics and risks are likely to affect multiple elements.
– “Implicitly trusting the most readily available information or anchoring too much
on convenient facts.” Russo and Schoemaker, 1990 - Decision trap # 5
Realistic cost analysis requires a systems engineering approach.
A credible cost analysis needs to integrate psychological findings
with mathematically valid models and sound management techniques.
E. K.., January 2005
Insight into structuring PCA
Consider n cost elements with uncertainty at WBS level-i
- Total project cost random variable
- Expect value
- Variance
e Consider Independent cost elements
- Fictitious reduction of uncertainty
Central limit theorem applies
- C
T
is a Gaussian normal distribution!
) ( ) (
?
=
n
j
j T C E C E
?
=
n
j
j T C C
1 ) , ( 1
)] ( * ) ( [ * ) , ( ) ( ) (
2 / 1
+ ? ? ?
+ =
? ??
?
j i
j i
j j j i
j i j T
C C Corr
C Var C Var C C Corr C Var C Var
n C E C Var T T / 1 ) ( / ) ( ?
Do not subdivide project cost into too many bite-size pieces!
Sum of 10 identical cost elements
?
?
E. K.., January 2005
Direct Fractile Assessment (DFA)
method
× Subjective assessment of cost elements
– DFA proven one of the most reliable and least bias-prone procedures for
eliciting uncertain quantities
· Experts provide 10
th
, 50
th
, and 90
th
percentiles
· Calibrate percentiles
– Default correction for optimism: 10
th
20
th
, 90
th
80
th
× Selection of realistic and flexible PDFs - Criteria
– C1. Fit 3 arbitrary percentiles
– C2. Finite lower range
– C3. Infinite upper range with reasonable behavior
– C4. Physically meaningful and easy to estimate parameters
The DFA method with calibration and
3-parameter Weibull can provide cost credibility
E. K.., January 2005
It’s the assessment!
× Values of the input percentiles
have a significant impact
× Important to select PDFs that fit
assessed percentiles
– Use Crystal Ball or @Risk
× Differences among fits where
expert opinion is unreliable
0%
20%
40%
60%
80%
100%
300 400 500 600 700 800 900
Cost , K$
C
o
s
t
p
r
o
b
a
b
i
l
i
t
y
o
f
s
u
c
c
e
s
s
W 10/50/90 W 20/50/80
B 10/50/90 B 20/50/80
Distribution
(fractiles*)
Mean
K$
SD^
K$
Weibull(10/50/90) 432 49
Weibull(20/50/80) 448 85
Beta(10/50/90) 432 45
Beta(20/50/80) 439 60
* 10/50/90: x
10
=382K$, x
50
=421K$, x
90
=499K$
20/50/80: x
20
=382K$, x
50
=421K$, x
80
=499K$.
E. K.., January 2005
MAIMS significantly impacts PCA cost
elements
Illustration- Cost element with 3-parameter Weibull distribution
Al l ocated
budget
X*, K$
Mean,
K$
Perc. of
mean
SD,
K$
Ideal 448 63 85
mean = 448 479 75 66
X50 = 422 463 72 73
X75 = 482 502 81 55
X85 = 521 535 87 48
4 Properties of MAIMS-Modified distributions
– Proper PDFs
– Minimum value: allocated budget, x*
– Modified Dirac delta function at x*
– Identical to original cost element for values > x*
¨ Not the same as Crystal Ball and @Risk truncated PDFs
MAIMS has a significant impact on PCA.
Impact increases with increased budget allocation.
E. K.., January 2005
A correlation model for dependencies at
the subsystem and system levels
4 There are multiple dependencies among cost elements
– Within a given subsystem due to technical complexity and common staff
– Among different subsystems due to common organizational and programmatic
considerations
¯ Consider cost elements C
m.j
- 1st and 2nd integers refer to WBS level 2 and level 3, respectively
» We model cost correlations based on Markowitz’s multi-factor model
C
m.j
= R
m.j
+ ?
m.j*
F
m
? ?
m.j
are constants; R
m.j
are independent random variables; F
m
are correlated random
variables; R
m.j
and F
n
are independent
? It can be shown Corr(C
m.j
, C
n.k
) = Corr(F
n
, F
m
)*?
m.j
*?
n.k
4 Given the lack of data, we make the following assumptions
- Simplified Two-Level Correlation Model (STLCM)
1. Corr(C
m.j
, C
m.k
) = ?
int
for cost elements in the same subsystem
2. Corr(C
m.j
, C
n.k
) = ?
ext
for cost elements in different subsystems
3. ?
int
> ?
ext
Important interrelationships in TLCM: system complexities,
staff, organizational and programmatic influences
E. K.., January 2005
Illustrative Analysis
Sample design/engineering project
WBS Cost Elements
Estimated Percentiles
K$
X10 X50 X90
1.0 Total project/system, CT
1.1 Project/system-level, C1
1.1.1 Project management, C1.1 382 421 499
1.1.2 Systems engineering, C1.2 220 232 257
1.1.3 Integration & test, C1.3 887 1,010 1,256
1.2 Subsystem X, C2
1.2.1 Mechanical components, C2.1 970 1088 1,323
1.2.2 Electrical components, C2.2 742 846 1,054
1.2.3 Integration & test, C2.3 596 724 980
1.3 Subsystem Y, C3
1.3.1 Software development, C3.1 1,069 1,282 1,708
1.3.2 Firmware, C3.2 634 743 961
1.3.3 Integration & test, C3.3 541 656 886
O Procedure
1. Establish CWBS
2. Assess cost elements
» Direct fractile assessment method
• 3 percentiles
• engineering judgment, experience,
& available data
3. Calibrate estimates
4. Fit estimates
» Three-parameter Weibull
5. Allocate budget to each cost element
6. Modify each cost element for MAIMS
7. Model correlation among cost elements
» Two-level correlation model
8. Perform Monte Carlo Simulation
9. Establish PoS
10. Determine total cost & contingency
» Modified PM approach
E. K.., January 2005
Budget allocation impacts project
cost and probability of success
4 Ideal Project
- “100% rational” team
- Each cost manager spends money only
as necessary to satisfy requirements
- Savings are available to support other
cost elements on an as-needed basis
· Actual costs may be less than
budgeted costs
4 Real Projects
- Human behavior and organizational
considerations
- MAIMS principle
- Budget and contingency management
are important confounding factors
- Effects increase with higher
allocated budgets and are
substantial
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
6,000 6,500 7,000 7,500 8,000 8,500 9,000 9,500
Cost, K$
MAIMS @ X50 MAIMS @ mean
MAIMS @ X75 Ideal
E. K.., January 2005
A tale of a project cost overrun
(1 of 2)
1. Agency X issues a RFP
- Requests cost at 50% CL
2. Contractor A prepares bid
© possesses limited sophistication;
but not cognizant of MAIMS
principle
- Develops CWBS
- Performs today’s typical PCA
• P50: 7,348 K$
• Min: 5,633 K$
3. Contractor A submits bid of
7,348 K$
? Confident he will succeed. Thinks
cost estimate has a 30% margin.
4. Contractor A is winner!
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
6,000 6,500 7,000 7,500 8,000 8,500 9,000 9,500
Cost, K$
MAIMS @X50 MAIMS @mean
MAIMS @X75 Ideal
E. K.., January 2005
A tale of a project cost overrun
(2 of 2)
5. The project starts & budgets are
allocated
- The practice is to baseline the Level-3
elements at mean values
• Baseline cost: 7,665 K$
® But project bid is 7,348 K$!
6. Much time is spent reallocating and
prorating budgets
- Budget cost elements at 50% CL
• Baseline cost: 7,002 K$
• Management reserve: ~ 5%
7. The outcome
® Everybody works very hard. But the
project runs out of budget and is
cancelled.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
6,000 6,500 7,000 7,500 8,000 8,500 9,000 9,500
Cost, K$
MAIMS @X50 MAIMS @mean
MAIMS @X75 Ideal
Epilogue
-Another project has succumbed to the MAIMS principle.
-Today’s typical PCA models a mythical project.
-Future RFPs, contracting agencies & contractors use proposed approach.
E. K.., January 2005
It’s NOT your classical contingency
anymore!
4 Cost contingency depends on desired probability of success and cost
management strategy
– MCC(PoS, PBC
1
,…,PBC
n
) = TEC(PoS, PBC
1
,…,PBC
n
) – PBC.
• MCC: Management Cost Contingency
• TEC: Total Estimated Cost
• PoS: Probability of Success
• PBC
i
: Baseline Budget for Cost element C
i
• PBC: sum over all cost elements.
» Management strategies and desired probabilities of success vary across business
categories
× Major differences with both deterministic practice and today’s typical PCA
» MCC is NOT a fixed percentage of PBC
» MCC incorporates MAIMS principle and depends on the management strategy
» Interactive and iterative process: system analysts, engineers, management
E. K.., January 2005
Realistic budget allocation, adequate contingency, and dynamic allocation
are critical to optimal cost and probability of success
Contingency, cost, & success
are NOT directly related
× High cost NEED NOT provide (1) high PoS or CL and/or (2) high contingency
× Low contingency DOES NOT necessarily equate to low cost
× High contingency DOES NOT necessarily equate to high cost and/or padding
5,500
6,000
6,500
7,000
7,500
8,000
8,500
9,000
9,500
10,000
10,500
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Confidence Level of Total Estimated Cost
T
o
t
a
l
E
s
t
i
m
a
t
e
d
C
o
s
t
,
K
$
0%
10%
20%
30%
40%
50%
60%
M
a
n
a
g
e
m
e
n
t
C
o
s
t
C
o
n
t
i
n
g
e
n
c
y
TEC MAIMS_@_mean TEC MAIMS_@_75 TEC MAIMS_@_50 TEC Ideal
MCC MAIMS_@_mean MCC MAIMS_@_75 MCC MAIMS_@_50
Different representation of data in previous figure
E. K.., January 2005
» Assessment of the cost elements
» Correlation effects
» Budget allocation
» Human behavior
» Organizational considerations
There are many confounding factors
to consider
These are important and confounding
factors that should be modeled
simultaneously in a PCA.
E. K.., January 2005
Our approach integrates many key concepts
Critical Chain
1
RACM
2
Today's Typical
PCA
Proposed Approach
Parameter Schedule Cost Cost Cost
Assessment of
uncertainty
- "Realistic" task
schedule = "Safe"
estimate/2
- Gaussian normal
PDFs
- Largely ad-hoc
- Extensive use of
triangular PDFs
- 3 percentiles using DFA
method
- 3-parameter Weibull
- Human
behavior
-Organizational
influences
- Parkinson' s law
- Safe estimates
- Multi-tasking
- MAIMS
- "Hidden"
incentives
"Ideal" project
& "100%
rational" person
- Calibrate cost elements
- Psychological findings
- MAIMS principle
Correlations Basic task
dependencies
None Limited, single
parameter model
Two-level correlation
model
Calculation
method
Deterministic, single-
point estimate
- Analytical/
statistical sum
Monte Carlo
simulation
Monte Carlo simulation
Project
management
- Project buffer
- Feeding buffers
- Project buffer: 25%
of original estimate
- Baseline budget
- Management
reserve
- Statistical cost
control
- Cost account
level and/or
management
reserve
- Baseline budget
- Management reserve
- Dynamic allocation
1
Goldratt's basic approach; numerous variations have been proposed
2
C. Gordon, Risk Analysis and Cost Management, Lockheed 1990's
E. K.., January 2005
Future directions
- Presented work focused on cost and macroscopic perspective
– it provides a framework for more accurate predictions
– it results in more realistic expectations
– benefits are likely to be significant
• more viable plans, better decisions, reduction in cost overruns.
× Much remains to be done
– integrate microscopic and macroscopic approaches
– simultaneously treat performance/cost/schedule
– quantitative calibration of data elicitation - single and multiple experts
Greatest challenge- implementation of systems thinking at the personnel,
organizational and institutional levels
- tool to dynamically adjust budget and modify negative behavior
- SE research to deal with psychological findings on human behavior and judgment
under uncertainty
Proposed approach is worth the additional effort!
doc_302300473.pdf