Description
In the risk mitigation phase, the organization identifies the types of controls that could be employed to reduce the level of risk to an acceptable level.
• Cognizant 20-20 Insights
Risk Mitigation: Fixing a Project Before It Is Broken
A comprehensive assessment of unforeseen risks in the project lifecycle can prevent costly breakdowns at the testing stage.
Executive Summary
Many IT projects have a reputation for exceeding budgets, missing deadlines, failing to realize expectations and/or delivering sub-par return on investment. Surveys and reports on the acceptability of new IT systems reinforce the same problems and probable causes of failure; yet businesses, large and small, continually make mistakes when attempting to improve information systems, particularly investing in inappropriate or unworkable changes without proper consideration of the likely risks. Planning projects, establishing requirements for change, selecting, implementing and testing suitable systems are important components of a superior business management process. Companies, however, often do not take adequate precautions during the project lifecycle to minimize failures at the testing stage. Consider the following: billion are wasted every year on badly managed IT system deployments, only discovered during the testing stage.1
• Only about 16% of overall IT projects can be
considered successful; even conservative estimates put the costs of failure into the £20 billion range across the European Union (EU), in IT software development projects.2
As seen in Figure 1 (next page), the peak expenditure of both time and money on IT support and development occurs at the testing stage. The costs of failure are not limited to the narrow considerations of budget and schedule. Companies do not pay adequate attention to conducting a proactive risk assessment of all projects during their entire lifecycle in an effort to mitigate and minimize the risk of failure at the testing stage. Identifying and analyzing potential risks of failure during the project lifecycle is fundamental to avoiding issues at the testing stage, as well as massive rework. To do this, it is vital to set up an early-stage diagnosis. This approach includes analyzing projects throughout the entire lifecycle, from the requirements gathering stage through verification, when all potential variables and intermediate process steps are weighed in terms of risk. Taking this approach can help IT organizations understand and minimize the potential impact on testing activities.
• Our
assessment of 134 publicly traded companies reveals that over half admitted to suffering a failed IT project in the past 12 months due to issues encountered during the testing stage. Such failures come at an extremely high cost. ing and the British Computer Society, £3.5
• According to the Royal Academy of Engineer-
cognizant 20-20 insights | july 2012
Project Costs Increase at Testing Stage
Rate of Cost Investment During Projected Lifecycle
New
Defect Prevention
Late Defect Discovery Results in Significant Rework
Requirements
Design and Build
Release to Test
Release to Field
Time
100x Increase in Cost of Removing Defects
Sources: Barry Boehm, “Software Engineering Economics,” Prentice Hall, Inc., 1981. Basili Boehm, “Software Management,” IEEE Computer, January 2001. Figure 1
In this whitepaper, we spell out our rigorous process for identifying and avoiding project issues that undermine many applications development projects. It also offers proven ways for organizations to minimize development delays and costs, while building systems that deliver exceptional business benefits.
Test Risk Assessment Process
Cognizant Business Consulting developed a structured, proprietary process called the Test Risk Assessment for Project Improvement (TeRAPI), with the main objective of identifying and minimizing risks that can occur at the testing stage across the entire product lifecycle. The process consists of three main phases: 1. Measure the business criticality. 2. Profile the risk level of projects. 3. Design a risk mitigation plan. The purpose of measuring business criticality is to determine the importance of each business function. After all the business processes are evaluated to determine their criticality to the project, business functions can be prioritized to determine which ones require a contingency plan and which can be ignored and eliminated. Each function, system, interface and third party can and should receive a risk rating. Probabilities are assigned to each failure that can occur
for each of these items. This means that an order function (risk rating = 5) could be impaired because of a business partner failure (probability of 60%) or could fail due to a system failure (probability of 20%). The same function would have a different criticality/probability score (see below), based on the fact that two different failures could hit that business function. The business criticality evaluation process is strictly related to the risk management process and is useful for any project stakeholder, at any project stage. Measuring Business Criticality To be successful, any test risk assessment has to concentrate on the local identifiable issues related to the business. During the test risk assessment process, risks to business functions will be identified and evaluated. The vulnerability of the business to these risks will be rated. Main actions to perform in a test risk assessment include:
• Alignment of testing with business
requirements. preparation.
• Test strategy and test environment • Test resourcing. • Test environment. • Test execution. • Production readiness preparation.
2
cognizant 20-20 insights
Each of these business functions represents a dimension to be analyzed in setting up a risk management process. Profiling Risk Level of Projects After having defined main dimensions and subdimensions as a core part of the TeRAPI methodology, each project must be assessed to weigh the risk associated with each sub-dimension in each stage of the project lifecycle. A business criticality score is determined based on the business purpose of the project. The result is a business risk index (BRI), which offers a composite view and comparison of risks based on the business criticality of the projects that require immediate management attention, as well as the priorities of these projects. All dimensions are quantitatively determined, from business requirements alignment, to test result analysis and mitigation plan design — and a qualitative index is assigned. Frequency of risk across each action step must be determined and graphed. For each dimension and its specific sub-dimensions, a qualitative measure of risk importance needs to be identified and quantified. This must be done across all projects where TeRAPI is applied. This data is placed in a qualitative table that summarizes all quantified risk-level results for
each assessed project in an effective way and in relation to each dimension and relative subdimensions (see Figure 2). The TeRAPI risk assessment identifies two types of risks for testing:
• High-level risks: Risks that are likely to impact
the schedule or the quality of the deliverable. The root causes for these risks need to be identified and addressed through initiatives at the higher level. risks: Risks that can be addressed by the testing team with minimal impact on schedule or quality. These risks can be addressed by project managers through tactical initiatives.
• Medium-level
Once the test risk assessment is completed, the next step is to analyze and present the results so that executive management can get the most use from the data. Data analysis will be the foundation for planning actions to mitigate risk impact and provide recommendations. The recovery strategies that need to be developed should be based on the findings of the risk assessment survey and interviews, as well as the business impact analysis. Designing the Risk Mitigation Plan Each TeRAPI dimension must be developed and risk causes determined and analyzed to create
TeRAPI: Project Test Risk Evaluation
Frequency of Risk for Individual Projects
2456 2620 2566 2761 2281 2881 2 1 4 2 7 1 1 4 2 1 1 0
10
TeRAPI DIMENSIONS Dimensions #1 Sub-dimension #1
Sub-dim ension #2 Sub-dim ension #3
PROJECTS
#1
H
H H
#2
H H
#3
#4
#5
M
#6
#7
H
H
#8
H
#9 #10 #11 #12 #13 #14 #15
H
M M
15 11 7 6 9 8 8 5 6 5 4 2 9 6 10
4
M
M
H H
M
M
M
M
Sub-dimension #4 Sub-dimension #5 Sub-dimension #6
H M
M M M
M M
M M M
M
M M M
Projects
2681 2847 2673 2710 2776c 2776b 2815 2912 2817
Sub-dimension #1 Sub-dimension #2 Sub-dimension #3 Sub-dimension #4 Sub-dimension #5 Sub-dimension #6 2 0 1
4 3 3 1 5 6 3
5
High Medium High (H) Risks that are likely to impact the schedule or quality of the deliverable Medium (M) Risks that can be addressed by the project manager with minimal impact on schedule and quality
5
10
15
20
2
4
6
8
10
Figure 2
cognizant 20-20 insights
3
a mitigation plan that addresses them. The mitigation plan, or risk response plan, provides options to reduce the likelihood that identified risks will occur. Options and actions are developed in a mitigation plan to enhance project opportunities and reduce threats to project objectives to “below an acceptable threshold.” Typically, every testing team maintains a mitigation plan, which consists of an Excel file containing different columns, including risk description, date, category, impact areas, root cause, priority, contingency, action plan, mitigation action plan, risk status and risk close date. Each mitigation action is related to the corresponding identified risk. Each action must be planned and time-lined in order to track it and update the probability that the corresponding risk will occur. The mitigation plan is managed by a supervisor team belonging to the testing organization and responsible for developing mitigation actions that will address all identified risks. To facilitate workload assignments, various workstreams can be created and assigned to different execution teams. If ranked risks are not mitigated properly, they will occur at certain points in time during testing activities. In that case, the risks become issues, and the supervisor team is responsible for identifying viable countermeasures and action plans to resolve them.
The supervisor team, which includes the sponsor, project manager, solution lead engineer, requirements engineer and test manager, is responsible for gathering the requests coming from the business and setting up an implementation plan in order to get them deployed according to the project plan. During the evaluation of the project scope, the team sets up a risk plan to evaluate potential constraints during solution implementation that would potentially occur during the requirements definition, solution development and code testing stages. For a tier one financial institution in Europe, two run-the-bank projects are considered, one without a risk management application, and the other with the TeRAPI application (see Figures 3 and 4). The project scope is to implement several change requests (CRs) for online private banking services. Both projects have a similar normalized initial budget estimated at CHF 400,000. The implementation of a structured risk assessment process, based on TeRAPI principles, has produced a cost savings of 19%, or around CHF 76,000, and reduced identified defects from 17 to 10 (about 60% less).
Before and After TeRAPI
Project 1 Initial budget Number of defects at testing stage Additional budget allocation to fix testing bugs Cost savings CHF 400,000 17 CHF 55,000 13.8% % Difference
TeRAPI Process Implementation: Business Case
A continuous risk management process is a necessary part of any approach to software security. In particular, at the testing stage, the primary objective is to improve the understanding of the major risks, in order to ensure stable and reliable testing. The TeRAPI approach can help test managers increase confidence in the tests of security function behaviors. Risk assessment is widely applied in all market industries and large organizations, but it is particularly important in banking and financial services. “Run-the-bank” projects implement a well-structured risk assessment process in order to manage all approved change requests by minimizing the risks during code testing. This process is set up at the scope identification stage.
– CHF 55,000 -13.8%
Project 2 Initial budget Number of defects at testing stage Additional budget allocation to fix testing bugs Cost savings Net cost savings
Figure 3
% Difference CHF 400,000 10 CHF 0 -58.8%
CHF 20,000 CHF 75,000
5.0% 18.8%
cognizant 20-20 insights
4
TeRAPI Implementation: Project Advantages
80 60 40 20 0 -20 -40 -60
Project 1 Project 2 Number of defects at testing stage Additional budget allocation Cost savings
Net cost savings
Figure 4
Project 1 has managed about 12 CRs without a risk management plan. It experienced 17 issues at the unit test stage, mainly related to not meeting requirements in code development. This resulted in a large amount of rework and the need for additional resources to cope with unexpected issues, increasing the project budget to CHF 55,000. In Project 2, the sponsor decided to use TeRAPI principles in an effort to reduce project expenditures, particularly during the testing stage. The supervisor team was established, and risks were identified. At the beginning, the risk was very large, with 50 identified risks. For each risk, a set of mitigation actions were established, with
a proper due date and responsibility assigned to members of the team. Prior to unit testing, the number of unaddressed risks was reduced to 10. During testing for the first release of the year, the number of issues to be addressed dropped by 60%, producing savings of CHF 20,000 compared with Project 1. The net savings obtained by Project 2 was about CHF 75,000. All identified risks were properly weighted to define risk relevance. The impact on the project implementation strategy, schedule, functionality, costs and quality were weighed, and each value concurred to define the risk probability.
TeRAPI: Risk Assessment Landscape
Top 15 Risks
100 75 Probability 50 25 0
20% 20% 60%
? Project Risk ? IT Risk ? Operational Risk
Risk Type
Low
Middle Impact
High
Risk Type (Open Risks) Project Risk IT Risk Operational Risk 3 1 1
Risk Category (Open Risks) Project Management Product Development Product (procured, operational) Sponsorship and Budget External Risks 1 2 1 1 0
20% 20%
Risk Category
20%
? Project Management ? Product Development ? Sponsorship and Budget ? External Risks
40%
Figure 5
cognizant 20-20 insights
5
TeRAPI: Risk Assessment Statement
Risk ID Keyword: Description: (Release Aug’12) - Unit tests execution warning Due to the project’s dependencies on external parties, there’s a threat that release CRs’ testing could be delayed, which will result into overall timeline shift
49
Cause/Origin : Project organizational structure involving multiple external parties Identification Date: 07-Jan-12 Test Manager
Consequences: Overall delay on release timeline Agreed functionalities during scope definition could not be delivered 100% Probability Risk (%) 30 Response Planning: Status: Open Closed on: Reason for Closure IT Risk Risk Type: Risk Category: Product Development Risk Owner: Last updated by: Last update on: Project Manager Project Manager 27-Mar-12 Costs: 10'000 (Risk) Risk Exposure: 3000.00 Project Area: Impact on Program Strategy 3 (1- 9) Response Action Impact on Project Schedule 7 (1- 9) Functionality 7 (1- 9) Costs 5 (1- 9) Quality 3 (1- 9) Cost Responsibility Type Response Due Date Effectiveness G A A A
1 Close coordination and weekly tracking with testing team and external support to verify and address identified test risks. 1'500 Test Manager 2 Unit test execution planning reviewed daily with external resources responsible. 500 Project and Test Managers 3 Regular followup with release management team and timely escalations. 1'000 Lead Engineer 1000 Project Manager 4 Pre-release testing to increase test coverage.
Mitigation 05-Mar-12 Mitigation 05-Mar-12 Mitigation 05-Mar-12 Mitigation 05-Mar-12
Project Management Risk Grade Program:
3 Risk Grade 2.10 Project:
0.90
Figure 6
The higher ranked risks were shown in a context that summarized all major risk properties, such as the probability to occur, risk type and category. Each risk was given a set of mitigation actions, with a relative supervisor team member assigned as risk owner and responsible for putting into practice all identified mitigation actions and tracking risk addressing. A risk assessment statement form (see Figure 6) collects all information concerning identified risk and is the kernel source for the final report.
The final report will take the following into consideration:
• Previous disruption history. • Risks and vulnerabilities. • Preventative measures. • Test risk assessment. • Mitigation plan.
The risk assessment process is an essential activity of continuity planning, in particular during the testing phase. Catching errors prior to testing the developed solution helps save time, money and effort and will result in cleaner, better performing and more business-aligned code. The TeRAPI process can enable agile and effective IT systems to support a more effective business. The purpose of the TeRAPI methodology during software testing is to identify high-risk applications that must be tested more thoroughly, as well as identifying error-prone components within specific applications that must be tested
TeRAPI Final Report
The risk landscape and mitigation plan will show a clear overview of risks, root causes and action agenda to resolve all challenges. The findings from TeRAPI will form the basis for the final report. A risk assessment outcome is the starting point for building the TeRAPI final report. The risk landscape shows the status of major risks to be tracked and reviewed periodically, trends across different reviews and the main categories to which the risks belong.
cognizant 20-20 insights
6
more rigorously. The results of the analysis can be used to determine the testing objectives for the application during test planning.
as the proper level of detail in prioritizing risks, at each stage of the project lifecycle. In order to implement this assessment process effectively, we estimate a team of three or four experts is needed to implement the process and prepare it for practical application for all involved projects. Our TeRAPI methodology is an effective way of minimizing risks during the testing stage, with the objective of minimizing delays and costs, and ensuring the highest business benefits. This approach provides companies with a consistent, business-based methodology for determining the level of risk present at the testing stage and how to mitigate it in order to deliver business value and the highest possible customer satisfaction.
Conclusion
Companies have always had to prioritize their testing efforts, but figuring out how to do it properly — with the right approach and methodology — has not always been clear. To compete, enterprises must reduce testing costs as low as possible without sacrificing application quality or delaying critical application launch time. TeRAPI can help in this regard. It is backed by statistical techniques and proven best practices gleaned from many years of experience with testing code of enterprises worldwide. Our approach ensures the appropriate amount of input from business and technical experts, as well
About the Authors
Simon Erdmann is a Director at Cognizant Business Consulting (CBC) in Frankfurt, Germany. He has over 15 years of experience in operations, management and consulting for the retail, consumer goods and transportation logistics industry, and is head of Germany, Austria and Switzerland for CBC Strategic Services. Simon has studied at German Armed Forces University and VWA Essen. He can be reached at [email protected]. Giuseppe L. Pannisco is a Consultant at Cognizant Business Consulting. He has six-plus years of consulting experience with global clients on supply chain management, IT strategy and management issues, working with companies in the automotive, aviation and aerospace, transportation and logistics service provider fields. Giuseppe holds a master’s degree in aerospace engineering (aeronautical business management specialization) from Universita’ degli Studi di Napoli “FEDERICO II.” He can be reached at [email protected].
cognizant 20-20 insights
7
About Cognizant
Cognizant (NASDAQ: CTSH) is a leading provider of information technology, consulting, and business process outsourcing services, dedicated to helping the world’s leading companies build stronger businesses. Headquartered in Teaneck, New Jersey (U.S.), Cognizant combines a passion for client satisfaction, technology innovation, deep industry and business process expertise, and a global, collaborative workforce that embodies the future of work. With over 50 delivery centers worldwide and approximately 140,500 employees as of March 31, 2012, Cognizant is a member of the NASDAQ-100, the S&P 500, the Forbes Global 2000, and the Fortune 500 and is ranked among the top performing and fastest growing companies in the world. Visit us online at www.cognizant.com or follow us on Twitter: Cognizant.
World Headquarters
500 Frank W. Burr Blvd. Teaneck, NJ 07666 USA Phone: +1 201 801 0233 Fax: +1 201 801 0243 Toll Free: +1 888 937 3277 Email: [email protected]
European Headquarters
1 Kingdom Street Paddington Central London W2 6BD Phone: +44 (0) 20 7297 7600 Fax: +44 (0) 20 7121 0102 Email: [email protected]
India Operations Headquarters
#5/535, Old Mahabalipuram Road Okkiyam Pettai, Thoraipakkam Chennai, 600 096 India Phone: +91 (0) 44 4209 6000 Fax: +91 (0) 44 4209 6060 Email: [email protected]
© Copyright 2012, Cognizant. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any
means, electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Cognizant. The information contained herein is subject to change without notice. All other trademarks mentioned herein are the property of their respective owners.
doc_203541197.pdf
In the risk mitigation phase, the organization identifies the types of controls that could be employed to reduce the level of risk to an acceptable level.
• Cognizant 20-20 Insights
Risk Mitigation: Fixing a Project Before It Is Broken
A comprehensive assessment of unforeseen risks in the project lifecycle can prevent costly breakdowns at the testing stage.
Executive Summary
Many IT projects have a reputation for exceeding budgets, missing deadlines, failing to realize expectations and/or delivering sub-par return on investment. Surveys and reports on the acceptability of new IT systems reinforce the same problems and probable causes of failure; yet businesses, large and small, continually make mistakes when attempting to improve information systems, particularly investing in inappropriate or unworkable changes without proper consideration of the likely risks. Planning projects, establishing requirements for change, selecting, implementing and testing suitable systems are important components of a superior business management process. Companies, however, often do not take adequate precautions during the project lifecycle to minimize failures at the testing stage. Consider the following: billion are wasted every year on badly managed IT system deployments, only discovered during the testing stage.1
• Only about 16% of overall IT projects can be
considered successful; even conservative estimates put the costs of failure into the £20 billion range across the European Union (EU), in IT software development projects.2
As seen in Figure 1 (next page), the peak expenditure of both time and money on IT support and development occurs at the testing stage. The costs of failure are not limited to the narrow considerations of budget and schedule. Companies do not pay adequate attention to conducting a proactive risk assessment of all projects during their entire lifecycle in an effort to mitigate and minimize the risk of failure at the testing stage. Identifying and analyzing potential risks of failure during the project lifecycle is fundamental to avoiding issues at the testing stage, as well as massive rework. To do this, it is vital to set up an early-stage diagnosis. This approach includes analyzing projects throughout the entire lifecycle, from the requirements gathering stage through verification, when all potential variables and intermediate process steps are weighed in terms of risk. Taking this approach can help IT organizations understand and minimize the potential impact on testing activities.
• Our
assessment of 134 publicly traded companies reveals that over half admitted to suffering a failed IT project in the past 12 months due to issues encountered during the testing stage. Such failures come at an extremely high cost. ing and the British Computer Society, £3.5
• According to the Royal Academy of Engineer-
cognizant 20-20 insights | july 2012
Project Costs Increase at Testing Stage
Rate of Cost Investment During Projected Lifecycle
New
Defect Prevention
Late Defect Discovery Results in Significant Rework
Requirements
Design and Build
Release to Test
Release to Field
Time
100x Increase in Cost of Removing Defects
Sources: Barry Boehm, “Software Engineering Economics,” Prentice Hall, Inc., 1981. Basili Boehm, “Software Management,” IEEE Computer, January 2001. Figure 1
In this whitepaper, we spell out our rigorous process for identifying and avoiding project issues that undermine many applications development projects. It also offers proven ways for organizations to minimize development delays and costs, while building systems that deliver exceptional business benefits.
Test Risk Assessment Process
Cognizant Business Consulting developed a structured, proprietary process called the Test Risk Assessment for Project Improvement (TeRAPI), with the main objective of identifying and minimizing risks that can occur at the testing stage across the entire product lifecycle. The process consists of three main phases: 1. Measure the business criticality. 2. Profile the risk level of projects. 3. Design a risk mitigation plan. The purpose of measuring business criticality is to determine the importance of each business function. After all the business processes are evaluated to determine their criticality to the project, business functions can be prioritized to determine which ones require a contingency plan and which can be ignored and eliminated. Each function, system, interface and third party can and should receive a risk rating. Probabilities are assigned to each failure that can occur
for each of these items. This means that an order function (risk rating = 5) could be impaired because of a business partner failure (probability of 60%) or could fail due to a system failure (probability of 20%). The same function would have a different criticality/probability score (see below), based on the fact that two different failures could hit that business function. The business criticality evaluation process is strictly related to the risk management process and is useful for any project stakeholder, at any project stage. Measuring Business Criticality To be successful, any test risk assessment has to concentrate on the local identifiable issues related to the business. During the test risk assessment process, risks to business functions will be identified and evaluated. The vulnerability of the business to these risks will be rated. Main actions to perform in a test risk assessment include:
• Alignment of testing with business
requirements. preparation.
• Test strategy and test environment • Test resourcing. • Test environment. • Test execution. • Production readiness preparation.
2
cognizant 20-20 insights
Each of these business functions represents a dimension to be analyzed in setting up a risk management process. Profiling Risk Level of Projects After having defined main dimensions and subdimensions as a core part of the TeRAPI methodology, each project must be assessed to weigh the risk associated with each sub-dimension in each stage of the project lifecycle. A business criticality score is determined based on the business purpose of the project. The result is a business risk index (BRI), which offers a composite view and comparison of risks based on the business criticality of the projects that require immediate management attention, as well as the priorities of these projects. All dimensions are quantitatively determined, from business requirements alignment, to test result analysis and mitigation plan design — and a qualitative index is assigned. Frequency of risk across each action step must be determined and graphed. For each dimension and its specific sub-dimensions, a qualitative measure of risk importance needs to be identified and quantified. This must be done across all projects where TeRAPI is applied. This data is placed in a qualitative table that summarizes all quantified risk-level results for
each assessed project in an effective way and in relation to each dimension and relative subdimensions (see Figure 2). The TeRAPI risk assessment identifies two types of risks for testing:
• High-level risks: Risks that are likely to impact
the schedule or the quality of the deliverable. The root causes for these risks need to be identified and addressed through initiatives at the higher level. risks: Risks that can be addressed by the testing team with minimal impact on schedule or quality. These risks can be addressed by project managers through tactical initiatives.
• Medium-level
Once the test risk assessment is completed, the next step is to analyze and present the results so that executive management can get the most use from the data. Data analysis will be the foundation for planning actions to mitigate risk impact and provide recommendations. The recovery strategies that need to be developed should be based on the findings of the risk assessment survey and interviews, as well as the business impact analysis. Designing the Risk Mitigation Plan Each TeRAPI dimension must be developed and risk causes determined and analyzed to create
TeRAPI: Project Test Risk Evaluation
Frequency of Risk for Individual Projects
2456 2620 2566 2761 2281 2881 2 1 4 2 7 1 1 4 2 1 1 0
10
TeRAPI DIMENSIONS Dimensions #1 Sub-dimension #1
Sub-dim ension #2 Sub-dim ension #3
PROJECTS
#1
H
H H
#2
H H
#3
#4
#5
M
#6
#7
H
H
#8
H
#9 #10 #11 #12 #13 #14 #15
H
M M
15 11 7 6 9 8 8 5 6 5 4 2 9 6 10
4
M
M
H H
M
M
M
M
Sub-dimension #4 Sub-dimension #5 Sub-dimension #6
H M
M M M
M M
M M M
M
M M M
Projects
2681 2847 2673 2710 2776c 2776b 2815 2912 2817
Sub-dimension #1 Sub-dimension #2 Sub-dimension #3 Sub-dimension #4 Sub-dimension #5 Sub-dimension #6 2 0 1
4 3 3 1 5 6 3
5
High Medium High (H) Risks that are likely to impact the schedule or quality of the deliverable Medium (M) Risks that can be addressed by the project manager with minimal impact on schedule and quality
5
10
15
20
2
4
6
8
10
Figure 2
cognizant 20-20 insights
3
a mitigation plan that addresses them. The mitigation plan, or risk response plan, provides options to reduce the likelihood that identified risks will occur. Options and actions are developed in a mitigation plan to enhance project opportunities and reduce threats to project objectives to “below an acceptable threshold.” Typically, every testing team maintains a mitigation plan, which consists of an Excel file containing different columns, including risk description, date, category, impact areas, root cause, priority, contingency, action plan, mitigation action plan, risk status and risk close date. Each mitigation action is related to the corresponding identified risk. Each action must be planned and time-lined in order to track it and update the probability that the corresponding risk will occur. The mitigation plan is managed by a supervisor team belonging to the testing organization and responsible for developing mitigation actions that will address all identified risks. To facilitate workload assignments, various workstreams can be created and assigned to different execution teams. If ranked risks are not mitigated properly, they will occur at certain points in time during testing activities. In that case, the risks become issues, and the supervisor team is responsible for identifying viable countermeasures and action plans to resolve them.
The supervisor team, which includes the sponsor, project manager, solution lead engineer, requirements engineer and test manager, is responsible for gathering the requests coming from the business and setting up an implementation plan in order to get them deployed according to the project plan. During the evaluation of the project scope, the team sets up a risk plan to evaluate potential constraints during solution implementation that would potentially occur during the requirements definition, solution development and code testing stages. For a tier one financial institution in Europe, two run-the-bank projects are considered, one without a risk management application, and the other with the TeRAPI application (see Figures 3 and 4). The project scope is to implement several change requests (CRs) for online private banking services. Both projects have a similar normalized initial budget estimated at CHF 400,000. The implementation of a structured risk assessment process, based on TeRAPI principles, has produced a cost savings of 19%, or around CHF 76,000, and reduced identified defects from 17 to 10 (about 60% less).
Before and After TeRAPI
Project 1 Initial budget Number of defects at testing stage Additional budget allocation to fix testing bugs Cost savings CHF 400,000 17 CHF 55,000 13.8% % Difference
TeRAPI Process Implementation: Business Case
A continuous risk management process is a necessary part of any approach to software security. In particular, at the testing stage, the primary objective is to improve the understanding of the major risks, in order to ensure stable and reliable testing. The TeRAPI approach can help test managers increase confidence in the tests of security function behaviors. Risk assessment is widely applied in all market industries and large organizations, but it is particularly important in banking and financial services. “Run-the-bank” projects implement a well-structured risk assessment process in order to manage all approved change requests by minimizing the risks during code testing. This process is set up at the scope identification stage.
– CHF 55,000 -13.8%
Project 2 Initial budget Number of defects at testing stage Additional budget allocation to fix testing bugs Cost savings Net cost savings
Figure 3
% Difference CHF 400,000 10 CHF 0 -58.8%
CHF 20,000 CHF 75,000
5.0% 18.8%
cognizant 20-20 insights
4
TeRAPI Implementation: Project Advantages
80 60 40 20 0 -20 -40 -60
Project 1 Project 2 Number of defects at testing stage Additional budget allocation Cost savings
Net cost savings
Figure 4
Project 1 has managed about 12 CRs without a risk management plan. It experienced 17 issues at the unit test stage, mainly related to not meeting requirements in code development. This resulted in a large amount of rework and the need for additional resources to cope with unexpected issues, increasing the project budget to CHF 55,000. In Project 2, the sponsor decided to use TeRAPI principles in an effort to reduce project expenditures, particularly during the testing stage. The supervisor team was established, and risks were identified. At the beginning, the risk was very large, with 50 identified risks. For each risk, a set of mitigation actions were established, with
a proper due date and responsibility assigned to members of the team. Prior to unit testing, the number of unaddressed risks was reduced to 10. During testing for the first release of the year, the number of issues to be addressed dropped by 60%, producing savings of CHF 20,000 compared with Project 1. The net savings obtained by Project 2 was about CHF 75,000. All identified risks were properly weighted to define risk relevance. The impact on the project implementation strategy, schedule, functionality, costs and quality were weighed, and each value concurred to define the risk probability.
TeRAPI: Risk Assessment Landscape
Top 15 Risks
100 75 Probability 50 25 0
20% 20% 60%
? Project Risk ? IT Risk ? Operational Risk
Risk Type
Low
Middle Impact
High
Risk Type (Open Risks) Project Risk IT Risk Operational Risk 3 1 1
Risk Category (Open Risks) Project Management Product Development Product (procured, operational) Sponsorship and Budget External Risks 1 2 1 1 0
20% 20%
Risk Category
20%
? Project Management ? Product Development ? Sponsorship and Budget ? External Risks
40%
Figure 5
cognizant 20-20 insights
5
TeRAPI: Risk Assessment Statement
Risk ID Keyword: Description: (Release Aug’12) - Unit tests execution warning Due to the project’s dependencies on external parties, there’s a threat that release CRs’ testing could be delayed, which will result into overall timeline shift
49
Cause/Origin : Project organizational structure involving multiple external parties Identification Date: 07-Jan-12 Test Manager
Consequences: Overall delay on release timeline Agreed functionalities during scope definition could not be delivered 100% Probability Risk (%) 30 Response Planning: Status: Open Closed on: Reason for Closure IT Risk Risk Type: Risk Category: Product Development Risk Owner: Last updated by: Last update on: Project Manager Project Manager 27-Mar-12 Costs: 10'000 (Risk) Risk Exposure: 3000.00 Project Area: Impact on Program Strategy 3 (1- 9) Response Action Impact on Project Schedule 7 (1- 9) Functionality 7 (1- 9) Costs 5 (1- 9) Quality 3 (1- 9) Cost Responsibility Type Response Due Date Effectiveness G A A A
1 Close coordination and weekly tracking with testing team and external support to verify and address identified test risks. 1'500 Test Manager 2 Unit test execution planning reviewed daily with external resources responsible. 500 Project and Test Managers 3 Regular followup with release management team and timely escalations. 1'000 Lead Engineer 1000 Project Manager 4 Pre-release testing to increase test coverage.
Mitigation 05-Mar-12 Mitigation 05-Mar-12 Mitigation 05-Mar-12 Mitigation 05-Mar-12
Project Management Risk Grade Program:
3 Risk Grade 2.10 Project:
0.90
Figure 6
The higher ranked risks were shown in a context that summarized all major risk properties, such as the probability to occur, risk type and category. Each risk was given a set of mitigation actions, with a relative supervisor team member assigned as risk owner and responsible for putting into practice all identified mitigation actions and tracking risk addressing. A risk assessment statement form (see Figure 6) collects all information concerning identified risk and is the kernel source for the final report.
The final report will take the following into consideration:
• Previous disruption history. • Risks and vulnerabilities. • Preventative measures. • Test risk assessment. • Mitigation plan.
The risk assessment process is an essential activity of continuity planning, in particular during the testing phase. Catching errors prior to testing the developed solution helps save time, money and effort and will result in cleaner, better performing and more business-aligned code. The TeRAPI process can enable agile and effective IT systems to support a more effective business. The purpose of the TeRAPI methodology during software testing is to identify high-risk applications that must be tested more thoroughly, as well as identifying error-prone components within specific applications that must be tested
TeRAPI Final Report
The risk landscape and mitigation plan will show a clear overview of risks, root causes and action agenda to resolve all challenges. The findings from TeRAPI will form the basis for the final report. A risk assessment outcome is the starting point for building the TeRAPI final report. The risk landscape shows the status of major risks to be tracked and reviewed periodically, trends across different reviews and the main categories to which the risks belong.
cognizant 20-20 insights
6
more rigorously. The results of the analysis can be used to determine the testing objectives for the application during test planning.
as the proper level of detail in prioritizing risks, at each stage of the project lifecycle. In order to implement this assessment process effectively, we estimate a team of three or four experts is needed to implement the process and prepare it for practical application for all involved projects. Our TeRAPI methodology is an effective way of minimizing risks during the testing stage, with the objective of minimizing delays and costs, and ensuring the highest business benefits. This approach provides companies with a consistent, business-based methodology for determining the level of risk present at the testing stage and how to mitigate it in order to deliver business value and the highest possible customer satisfaction.
Conclusion
Companies have always had to prioritize their testing efforts, but figuring out how to do it properly — with the right approach and methodology — has not always been clear. To compete, enterprises must reduce testing costs as low as possible without sacrificing application quality or delaying critical application launch time. TeRAPI can help in this regard. It is backed by statistical techniques and proven best practices gleaned from many years of experience with testing code of enterprises worldwide. Our approach ensures the appropriate amount of input from business and technical experts, as well
About the Authors
Simon Erdmann is a Director at Cognizant Business Consulting (CBC) in Frankfurt, Germany. He has over 15 years of experience in operations, management and consulting for the retail, consumer goods and transportation logistics industry, and is head of Germany, Austria and Switzerland for CBC Strategic Services. Simon has studied at German Armed Forces University and VWA Essen. He can be reached at [email protected]. Giuseppe L. Pannisco is a Consultant at Cognizant Business Consulting. He has six-plus years of consulting experience with global clients on supply chain management, IT strategy and management issues, working with companies in the automotive, aviation and aerospace, transportation and logistics service provider fields. Giuseppe holds a master’s degree in aerospace engineering (aeronautical business management specialization) from Universita’ degli Studi di Napoli “FEDERICO II.” He can be reached at [email protected].
cognizant 20-20 insights
7
About Cognizant
Cognizant (NASDAQ: CTSH) is a leading provider of information technology, consulting, and business process outsourcing services, dedicated to helping the world’s leading companies build stronger businesses. Headquartered in Teaneck, New Jersey (U.S.), Cognizant combines a passion for client satisfaction, technology innovation, deep industry and business process expertise, and a global, collaborative workforce that embodies the future of work. With over 50 delivery centers worldwide and approximately 140,500 employees as of March 31, 2012, Cognizant is a member of the NASDAQ-100, the S&P 500, the Forbes Global 2000, and the Fortune 500 and is ranked among the top performing and fastest growing companies in the world. Visit us online at www.cognizant.com or follow us on Twitter: Cognizant.
World Headquarters
500 Frank W. Burr Blvd. Teaneck, NJ 07666 USA Phone: +1 201 801 0233 Fax: +1 201 801 0243 Toll Free: +1 888 937 3277 Email: [email protected]
European Headquarters
1 Kingdom Street Paddington Central London W2 6BD Phone: +44 (0) 20 7297 7600 Fax: +44 (0) 20 7121 0102 Email: [email protected]
India Operations Headquarters
#5/535, Old Mahabalipuram Road Okkiyam Pettai, Thoraipakkam Chennai, 600 096 India Phone: +91 (0) 44 4209 6000 Fax: +91 (0) 44 4209 6060 Email: [email protected]
© Copyright 2012, Cognizant. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any
means, electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Cognizant. The information contained herein is subject to change without notice. All other trademarks mentioned herein are the property of their respective owners.
doc_203541197.pdf