Description
NASA carries out a variety of quantitative risk assessments in conducting its work. These assessments range from specialized studies, such as the assessment of the safety risk involved in transfer of the DC-8 aircraft, to large scale probabilistic risk assessments (PRAs), such as the PRAs conducted on the Space Station and Space Shuttle.
Quantitative Risk Analysis: Challenges and Opportunities at NASA
Abstract
NASA carries out a variety of quantitative risk assessments in conducting its work. These
assessments range from specialized studies, such as the assessment of the safety risk
involved in transfer of the D!" aircraft, to large scale pro#a#ilistic risk assessments
$%&As', such as the %&As conducted on the Space Station and Space Shuttle. (ecause of
the range of pro#lems addressed, there are a various challenges in carrying out
quantitative risk assessments involving #oth modeling and implementation. There are
also opportunities for using quantitative risk analysis to improve decision making and
risk management. This paper presents a spectrum of e)amples of quantitative risk
analyses, from large scale to specialized. The e)amples also illustrate the range of
approaches #eing utilized or #eing developed. The e)amples are the Space Shuttle %&A, a
decision application accounting for risk uncertainties, a pro*ect risk assessment of
cumulative risks, and an assessment of software development risks.
Introduction
NASA conducts a variety of quantitative risk assessments to support pro*ect management
and decision making. +,uantitative risk assessments- means here assessments that
estimate numerical values for pro#a#ilities and consequences, along with associated
uncertainties. .any assessments focus only on estimating pro#a#ilities of failure or
pro#a#ilities of other undesira#le events occurring. These are also considered as #eing
quantitative risk assessments here. The quantitative risk assessments that are carried out
range from narrow, specialized relia#ility or pro#a#ilistic assessments to large scale
%ro#a#ilistic &isk Assessments $%&As'. %ro*ect managers utilize these quantitative risk
assessments, as deemed appropriate, in assessing and managing risk.
/ven though a variety of quantitative risk assessments are performed, there is presently
no standard framework for carrying out and for utilizing quantitative risk results. &isk
matrices are standardly used in pro*ect risk management, such as the traditional 0)0 risk
matri), which categorizes a contri#utor1s pro#a#ility and consequence into a given
category. 2owever, these are generally viewed as #eing qualitative risk assessments, with
sometimes minimal underlying, structured quantitative risk assessments carried out.
There is thus a challenge and opportunity for upgrading the role of quantitative risk
assessments to #etter assist pro*ect management. This paper presents a spectrum of
e)amples that span different approaches and applications. The e)amples are the Space
Shuttle %&A, a decision application accounting for risk uncertainties, a pro*ect risk
assessment of cumulative risks, and an assessment of software development risks.
PRA at NASA
At NASA, %ro#a#ilistic &isk Assessments, or %&As, are conducted in various situations
to quantify the associated risk. The most e)tensive %&A conducted is a full scope %&A,
which models a comprehensive set of scenarios leading to all undesired end states , such
as loss of crew and potential in*ury to the pu#lic. A full scope %&A models the
comprehensive set of initiating events and contri#utors that can lead to the undesired end
states. The #asic causes that are modeled include hardware failures, human errors,
%age 3 of 3"
process errors, and phenomenological events. The ne)t lower level %&A is a limited
scope %&A, which focuses on one or more specific end states such as a specific mission
failure. redi#le initiating events and contri#utors leading to the particular end states are
included. The lowest level %&A is a simplified scope %&A focusing on a particular end
state and including only particular contri#utors.
NASA1s %rocedural &equirement N%& "450.0 $3' defines when a %&A is required and the
scope of %&A required. 6n general a full scope %&A is required for new programs when
the consequences can involve human safety or health or can entail high schedule
criticality. 7imited scope %&As or simplified scope %&As are required or recommended
in other situations. A limited scope %&A or simplified scope %&A can #e waived at the
direction of the program or pro*ect with sufficient #asis. The ta#le on the ne)t page
summarizes the criteria for the scope of %&A required in new NASA pro*ects. .ore
details are contained in NASA1s N%& "450.0.
The Space Shuttle PRA
The Space Shuttle %&A, has recently #een completed and is #eing revised and #eing
updated. This is actually the most recent Space Shuttle %&A since several earlier %&As
have #een carried out on the Space Shuttle. This most recent %&A is most nota#le for its
comprehensiveness, detail, and involvement #y the NASA centers The Space Shuttle
%&A is like any other traditional, full scale %&A in that it constructs accident scenarios
that identify events and failures that can lead to a defined end state or end states. 8or the
Space Shuttle %&A, the end state that is the focus is the loss of crew and vehicle $79:'
during a mission. The accident scenarios are modeled using event trees. /ach accident
scenario that contains one or more system failures generally uses fault trees to resolve the
#asic events leading to system failure. .any accident sequences involve only one event
where that event is a loss of critical tile, a critical de#ris hit, an e)plosion or fire, or other
phenomena. The #asic events, which in certain scenarios also involve crew errors or
component functional failures, are then quantified to determine the pro#a#ilities of the
accident scenarios occurring. 8igure 3 illustrates the event tree and fault tree modeling in
a %&A. NASA1s %&A %rocedure ;uide $NS!
3555 per launch. The second step is then
to determine the mean failure pro#a#ility value to compare with the criterion. This
involves using agreed!upon models, data, and uncertainty characterizations. The
uncertainty characterizations are used to determine the uncertainty distri#ution for the
%age E of 3"
estimated 9%: failure pro#a#ility per mission from the
NASA carries out a variety of quantitative risk assessments in conducting its work. These assessments range from specialized studies, such as the assessment of the safety risk involved in transfer of the DC-8 aircraft, to large scale probabilistic risk assessments (PRAs), such as the PRAs conducted on the Space Station and Space Shuttle.
Quantitative Risk Analysis: Challenges and Opportunities at NASA
Abstract
NASA carries out a variety of quantitative risk assessments in conducting its work. These
assessments range from specialized studies, such as the assessment of the safety risk
involved in transfer of the D!" aircraft, to large scale pro#a#ilistic risk assessments
$%&As', such as the %&As conducted on the Space Station and Space Shuttle. (ecause of
the range of pro#lems addressed, there are a various challenges in carrying out
quantitative risk assessments involving #oth modeling and implementation. There are
also opportunities for using quantitative risk analysis to improve decision making and
risk management. This paper presents a spectrum of e)amples of quantitative risk
analyses, from large scale to specialized. The e)amples also illustrate the range of
approaches #eing utilized or #eing developed. The e)amples are the Space Shuttle %&A, a
decision application accounting for risk uncertainties, a pro*ect risk assessment of
cumulative risks, and an assessment of software development risks.
Introduction
NASA conducts a variety of quantitative risk assessments to support pro*ect management
and decision making. +,uantitative risk assessments- means here assessments that
estimate numerical values for pro#a#ilities and consequences, along with associated
uncertainties. .any assessments focus only on estimating pro#a#ilities of failure or
pro#a#ilities of other undesira#le events occurring. These are also considered as #eing
quantitative risk assessments here. The quantitative risk assessments that are carried out
range from narrow, specialized relia#ility or pro#a#ilistic assessments to large scale
%ro#a#ilistic &isk Assessments $%&As'. %ro*ect managers utilize these quantitative risk
assessments, as deemed appropriate, in assessing and managing risk.
/ven though a variety of quantitative risk assessments are performed, there is presently
no standard framework for carrying out and for utilizing quantitative risk results. &isk
matrices are standardly used in pro*ect risk management, such as the traditional 0)0 risk
matri), which categorizes a contri#utor1s pro#a#ility and consequence into a given
category. 2owever, these are generally viewed as #eing qualitative risk assessments, with
sometimes minimal underlying, structured quantitative risk assessments carried out.
There is thus a challenge and opportunity for upgrading the role of quantitative risk
assessments to #etter assist pro*ect management. This paper presents a spectrum of
e)amples that span different approaches and applications. The e)amples are the Space
Shuttle %&A, a decision application accounting for risk uncertainties, a pro*ect risk
assessment of cumulative risks, and an assessment of software development risks.
PRA at NASA
At NASA, %ro#a#ilistic &isk Assessments, or %&As, are conducted in various situations
to quantify the associated risk. The most e)tensive %&A conducted is a full scope %&A,
which models a comprehensive set of scenarios leading to all undesired end states , such
as loss of crew and potential in*ury to the pu#lic. A full scope %&A models the
comprehensive set of initiating events and contri#utors that can lead to the undesired end
states. The #asic causes that are modeled include hardware failures, human errors,
%age 3 of 3"
process errors, and phenomenological events. The ne)t lower level %&A is a limited
scope %&A, which focuses on one or more specific end states such as a specific mission
failure. redi#le initiating events and contri#utors leading to the particular end states are
included. The lowest level %&A is a simplified scope %&A focusing on a particular end
state and including only particular contri#utors.
NASA1s %rocedural &equirement N%& "450.0 $3' defines when a %&A is required and the
scope of %&A required. 6n general a full scope %&A is required for new programs when
the consequences can involve human safety or health or can entail high schedule
criticality. 7imited scope %&As or simplified scope %&As are required or recommended
in other situations. A limited scope %&A or simplified scope %&A can #e waived at the
direction of the program or pro*ect with sufficient #asis. The ta#le on the ne)t page
summarizes the criteria for the scope of %&A required in new NASA pro*ects. .ore
details are contained in NASA1s N%& "450.0.
The Space Shuttle PRA
The Space Shuttle %&A, has recently #een completed and is #eing revised and #eing
updated. This is actually the most recent Space Shuttle %&A since several earlier %&As
have #een carried out on the Space Shuttle. This most recent %&A is most nota#le for its
comprehensiveness, detail, and involvement #y the NASA centers The Space Shuttle
%&A is like any other traditional, full scale %&A in that it constructs accident scenarios
that identify events and failures that can lead to a defined end state or end states. 8or the
Space Shuttle %&A, the end state that is the focus is the loss of crew and vehicle $79:'
during a mission. The accident scenarios are modeled using event trees. /ach accident
scenario that contains one or more system failures generally uses fault trees to resolve the
#asic events leading to system failure. .any accident sequences involve only one event
where that event is a loss of critical tile, a critical de#ris hit, an e)plosion or fire, or other
phenomena. The #asic events, which in certain scenarios also involve crew errors or
component functional failures, are then quantified to determine the pro#a#ilities of the
accident scenarios occurring. 8igure 3 illustrates the event tree and fault tree modeling in
a %&A. NASA1s %&A %rocedure ;uide $NS!
3555 per launch. The second step is then
to determine the mean failure pro#a#ility value to compare with the criterion. This
involves using agreed!upon models, data, and uncertainty characterizations. The
uncertainty characterizations are used to determine the uncertainty distri#ution for the
%age E of 3"
estimated 9%: failure pro#a#ility per mission from the