Description
Evaluation is a systematic determination of a subject's merit, worth and significance, using criteria governed by a set of standards. It can assist an organization, program, project or any other intervention or initiative to assess any aim, realisable concept/proposal, or any alternative, to help in decision-making; or to ascertain the degree of achievement or value in regard to the aim and objectives and results of any such action that has been completed.
KOELMANS, R.G. Project success and performance evaluation. International Platinum Conference ‘Platinum Adding Value’, The South African Institute of Mining and Metallurgy, 2004.
Project success and performance evaluation
R.G. KOELMANS
Read, Swatman & Voigt (Pty) Limited
In 2001 Platinum overtook gold in foreign revenue earnings for South Africa. The export sales from platinum group metals amounted to R29.3 billion as opposed to gold export sales of R28.6 billion1. Organizations must undertake new ventures to replace old, worn-out and obsolete assets with new ones. These assets could be a building, a new product or process, a new mine, or anything that can help the organization to achieve its objective. These ventures are projects by their nature. When a project is pronounced a success, the judgement is usually based on some factual evidence, although not every one uses the same data. From the literature, it is obvious that there is a lack of consensus and objectivity as to what constitutes a successful project. The purpose of this paper is to address the following questions: • What constitutes a successful project? • How does one measure project success? • How can one work toward project success? This paper addresses the requirements for project success of three of the primary stakeholders of a project, namely the business or client, the employees as far as the project team is concerned, and the local community. Indicators are developed into a work breakdown structure (WBS)-like chart to demonstrate success indicators. These indices take cognisance of all the constituents required for project success. The indicators are further developed to take cognisance of the requirements of the local communities. These indices are then weighted to arrive at a total score for the respective stakeholders. Using these WBS models, the success and performance of a project can be evaluated. This score is considered an objective measure of project success. Projects are evaluated more consistently across different projects and the project manager is provided with a management tool that can assist him/her in ensuring greater project success. The shortcoming of this approach is that the weights of the different indices have been estimated at this stage. Research will have to be conducted to more accurately reflect the weights assigned to the indices.
Introduction
Despite the significant development of project management principles in recent years, there are still a large proportion of projects that fail, or stated differently, projects that are not classed as being a success. Many projects have been and will still be undertaken in the platinum industry, be it expansion projects or process change projects. It makes sense then, unless some form of intervention occurs, that a large number of future projects could still be unsuccessful. Every one involved with a project wants it to be a success. Many cases can be cited from literature and anecdotal data of projects that fall short on expectations in one or more of the triple constraint items (time, cost and performance), or in terms of client satisfaction, and yet the project team officially pronounces the project a success. Success can mean different things to different people. An interesting example of this is provided: ‘An architect may consider success in terms of aesthetic appearance, an engineer in terms of technical competence, an accountant in
terms of dollars spent under budget, a human resources manager in terms of employee satisfaction. Chief executive officers rate their success in the stock market.’2 Freeman and Beale1 cite the fact that the motivation of their paper stems from the lack of consensus and objectivity as to what constitutes a successful project. This raises the following questions: • What constitutes a successful project? • How does one measure project success? • How can one work toward greater project success? The requirements of each stakeholder group will differ and therefore their perception of what constitutes success will vary. This paper analyses the requirements of three primary stakeholders and how these are measured. The requirements of some of these primary project stakeholders will initially be addressed, as well as project requirements, the goals of a project system, project constraints and what constitutes a successful project.
PROJECT SUCCESS AND PERFORMACE EVALUATION
229
Finally, this paper explores the development of a set of indices that can be used to evaluate project performance and success to make explicit what is implicit in seemingly subjective evaluations.
Project stakeholders
A stakeholder by definition is any one that has a stake in the project or its outcome. Stated differently, it is any interested party in the project or its outcome. Business stakeholders can be divided into primary and secondary and further subdivided into social and non-social stakeholders. Carroll and Buchholtz3 list the shareholders and investors (owners), employees and managers, customers, local communities, suppliers and other business partners as primary social stakeholders. When a business embarks on a project, a number of stakeholders can be identified and the list is by no means exhaustive: • The project sponsor and client • Owners and shareholders • Employees of the parent company undertaking the project • The community at large • The project team involved in the project • National and local governments • Suppliers who could be supplying products required by the project • The contractors who could benefit by working on the execution of the project • Competitors who could have a keen interest in the project as it may affect how they will operate in the future • Consumers. From these stakeholders, the paper will address the requirements for project success of some of the primary stakeholders, namely the business or client, the employees as far as the project team is concerned, and the local community.
There could be an even greater difference and variation in the goals of the community. In the South African context, the provision of additional jobs, social development and community involvement will rank high on the agenda of goals. Other objectives could be low noise and pollution levels. Increases in traffic volumes and noise could also be problematic to the community. Whether the scope of the project is for a product or constructed item, or a service, it will be translated into goals and objectives. These goals will be different for each stakeholder group but can be split into tangible goals such as time, cost, quality, and utility and intangible goals such as stakeholder satisfaction.
Constraints
A project can be viewed as an input-output model. Some form of input is transformed into output, under a set of constraints and utilizing a set of mechanisms to make the project happen3. The main constraints are time, cost and quality for the following reasons: • Time. Most if not all projects have a time constraint • Cost. Money to fund the project is a scarce resource. Clients want value for money • Quality. This is the standard by which the product, the end result of the project, and the process will be judged. In addition to the above main constraints, the following constraints can be limiting on the project4: • Legal issues have to be complied with • Ethical issues • Environmental issues • Logic—the need for certain activities to be completed before a project can start • Indirect effects—any change is most likely to have a ripple effect, which has to be taken into account. Trade-offs are invariably required to balance the constraints, especially the three main constraints. For example, a task may be completed very quickly through the addition of more resources, but unless one is prepared to compromise on quality, one will have to pay more. These trade-off decisions occur both during the planning and execution phase of the project4.
Project requirements
A project is started to fulfil a need or requirement. This requirement can be in the form of something tangible such as a new shaft system to access and exploit ore reserves, an improvement to an existing product such as a development waste loader, etc. or intangible such as developing a new mining method, a downsizing exercise, etc. A project is a temporary endeavour to create a unique product or service, has a start and end and is non-repetitive. These characteristics differentiate projects from operations4. The basic purpose of a project is to accomplish a specific goal5.
Measurement of project success
Projects are becoming more complex with the passage of time. There are a number of reasons why: • The simpler ideas have been exploited first and the need for innovation is vital • The allowable time to market is reducing, which impacts on one of the important constraints of all projects • Businesses are becoming more complex • Projects are also moving towards turnkey contracts, where the end user does not get involved with the interface between the parts of a system, but only deals with a single supplier in the provision of the entire system4. A vast number of articles and papers has been written and published on projects that have failed, sometimes with disastrous consequences. The explosion of the space shuttle, Challenger, just after take-off on 28 January 1986 and the more recent collapse of the Bushbuck Ridge bridge in Mpumalanga are examples of such major project failures. Many more can be cited.
Goals and objectives of a project system
The goals of each of the stakeholder groups will differ and vary according to the nature of the project. Generally, the client wants the required project with the required performance (quality and utility) for the lowest cost and shortest time. The scope of the project spells out what is to be done, by when and how much the costs will be. The project team goals will vary in urgency and sophistication. They could include to set industry standards, to integrate project management into the organization, to make a certain profit for the project management office, or to simply finish the project on time and within budget6.
230
PLATINUM ADDING VALUE
In market-driven economies organizations are established to supply goods or services to consumers. A private organization differs from a public one in that the former has a profit motive realized by exploiting competitive advantages in the market. Organizations must undertake new ventures to replace old, worn-out and obsolete assets with new ones. These assets could be a building, a new product or process, a new mine, or anything that can help the organization to achieve its objective7. These ventures are projects by their nature. A project by definition is a unique and temporary endeavour undertaken to create a product or service8. Anyone involved with a project wants it to be a success. The issue of project success is frequently discussed, yet ideas of what constitutes a successful project are many and varied. Pinto and Slevin9 cite that many people are aware of projects that come in on time and under budget and were nevertheless considered failures, yet the opposite is equally true. Similarly, Rad and Levin6 state that ‘many cases can be cited from the literature and anecdotal data of projects that fall short on expectations in one or more of the triple constraint items (time, cost and quality), or in terms of client satisfaction and yet the project team officially announces the project a success.’ Success can mean different things to different people. An interesting example of this has been provided: ‘An architect may consider success in terms of aesthetic appearance, an engineer in terms of technical competence, an accountant in terms of dollars spent under budget, a human resources manager in terms of employee satisfaction. Chief executive officers rate their success in the stock market.’2 Each project has different and unique stakeholders who have their own specific requirements for a project and therefore their own measure of what good project performance is and finally what a successful project entails. The further removed from the project, the more obscure these objectives and requirements may be. Different people, even if they are part of the same organization, will view success in a different light at different times10. According to Gido and Clements 11 , project success consists of four components namely budget (costs), schedule (time), performance (quality and utility), and customer satisfaction. The key to project success is the people, the project team and their organization (project management office), the tools and techniques used by the project team and the understanding the team has of the requirements and agendas of the stakeholders. One further aspect that has recently come to the fore is the question of the project’s impact on health, safety and the environment12. These combined criteria, factors and key issues can be grouped together as the constituents for project success and can be depicted as in Figure 1. Freeman and Beale1 cite the fact that the motivation for their paper stems from the lack of consensus and objectivity as to what constitutes a successful project. Project success is the most debated topic in the management field, but also the least agreed upon10. Discussion of project failures is never easy for the organizations concerned, but anecdotal data provide many examples of where a project falls short on one or more of the iron triangle items (time, cost and performance) or in terms of client satisfaction, but the team officially pronounce it a success. The converse has also occurred. The team may consider the project a success, while the client declares it a failure. The pronouncement of success or
failure may not even be unanimous among the team and or client personnel6. When a project is pronounced as a success, the judgment is usually based on some factual evidence, although not every one uses the same data. Even if different people use the same data, the same set of evaluation indices are not used in arriving at the degree of success of a project. The perception of success is sometimes based on feelings and personal indices5. The relationship between satisfaction, expectation and perception is quite simple. Satisfaction is determined by the difference between how the project is perceived or viewed by a stakeholder and how they expect the project to perform4. Satisfaction = perception - expectation From the above it is obvious that ‘thing-related’ issues such as data and ‘people-related’ issues such as perceptions play a role in whether a project is considered a success by the different stakeholders. How does one then measure project success? Success indicators and factors The three primary stakeholders in question will have different viewpoints of the success of a project. The client will be focused on the outcomes of the project, the deliverables. The focus will be on the goals and objectives of the project, more especially on the scope and quality aspects of the deliverables. In certain instances, time and costs issues are of secondary importance but could be just as important as the former aspects. The scope and quality of the project has been identified as the most influential factor among the elements of the iron triangle. The performance in cost and time (schedule) will be measured, similar to scope, against the final values and the relative magnitude of the variances. Certain variances can be justified, but the unjustified variance will influence the judgment whether the project was a success or not. These items lend themselves to quantification and therefore these indices lend themselves to mathematical precision. Independent of the above, the perception of success can be influenced by issues such as responsiveness of the team to client requests for changes, punctuality at meetings, communication or the lack thereof, personality clashes and
n tio ac isf at ts ien Cl
Utility
Sc he du le
Health, safety and environment
Quality
d an es ls iqu o n To ech t
Budge t
m tea ect e and ) j o Pr eopl ation (p aniz org
Figure 1. A schematic representation of the constituents of a successful project
PROJECT SUCCESS AND PERFORMACE EVALUATION
231
other people issues. These issues can be grouped into team morale and client satisfaction, and are harder to measure and quantify. The resulting quantity is usually subjective and can be open to lengthy debate6. Rad and Levin propose that these indicators be developed into a work breakdown structure (WBS)-like chart to demonstrate client success indicators 5 . A WBS is a breakdown of the project or work into smaller packages. The project is therefore subdivided into hierarchical units of tasks (level 1), subtasks (level 2), work packages (level 3) and so on. The WBS as proposed by Rad and Levin6 for both the client and the team takes cognizance of all the constituents as depicted in Figure 1 except for the safety, health and environmental issues. It can be argued that the requirements for these three issues form part of the scope, but the author has considered these items important enough to separate them. This WBS has been developed and unpacked for a project management and consulting engineering firm doing various projects for the platinum mines. This WBS is depicted in Figure 2. The project team, in turn, will be focused on the final deliverables and the process by which the deliverables are created. Included in the process are the tools and techniques used by the organization to execute its work. In order for a project to be considered a success by the team, all the activities indicated by the team factors must be managed well, regardless of the phase and of changes to scope, schedule and costs. The major difference between client success indicators and team success factors is that client indicators focus largely on the performance characteristics and external appearances of the product. The team success factors, on the other hand, focus on activities and measures that produce the deliverables of the project5. The project
team wants to deliver the project using best practices and procedures, which not only makes the team operate effectively, but also ensures a project that meets the required performance criteria and objectives of the client. In order to assess the success and effectiveness of the project manager and the team in the execution of their project, one has to identify three separate elements in the management process. The first is the existence of standardized procedures to do the work, the team’s conformity with these procedures, and finally how efficient these procedures make the work of the team. Project success factors can again be divided into two major categories similar to the client view, issues that deal with things and those issues that deal with people. In practice the things issues are tangible but intertwined with the people, the intangible issues. The degeneration of the people issues will have an impact on the things issues7. The next level of items that form the basis for project success factors are scope, cost, quality and schedule. According to Rad and Levin6, there are, however, items, which are things related but are also influenced by people to a certain degree. These are contract administration, project integration, reporting and risk. The management of these items requires documents, charts and reports (things) and also dealings with people, more so than scope, cost and quality. The management of the aforementioned items therefore contains an element of people issues. Level two of people issues include the management of the team issues, client issues, supplier and contractor issues and the various forms of communication. The relative weight placed on each of these issues will be influenced by the organization’s objectives, the type of project and project phase as shown in Figure 3.
Scope as needed
Quality as needed Things-related attributes Schedule on time Project success indicators
Cost within budget
Team morale People-related attibutes Client satisfaction
Figure 2. Project success indicators: client view5 (adapted)
232
PLATINUM ADDING VALUE
Scope Quality Things-related issues Schedule Cost Safety, health and environmental Integration Project success indicators Things-related issues Reporting Risk management Contract administration Team Suppliers & contractors Client Communication
People-related issues
Figure 3. Project success factors: team view5 (adapted)
For both the client and the project team, the aspects that determine a successful project, as viewed by the community, can be grouped into attributes related to things and those related to people. Issues that can be categorized as things would be the environmental impact and long-term sustainability of the project’s outcome. The people issues are considered to be safety, economic and social contribution, job creation, Black Economic Empowerment and affirmative action. Environmental issues are usually one of the first concerns of the community when a new project is launched. For this reason, legislation requires that an environmental impact assessment be carried out to ensure that mitigating actions are put in place, should there be a threat to the environment. Pollution, in all its forms (air, noise and water) is high on the community’s list of priorities. Energy consumption for example, can have a big impact on water consumption and air pollution through the generation of coal fired electricity. Legislation requires certain preventive measures to be in place to protect the environment. Often these requirements are barely sufficient as a minimum and when these projects are designed, such requirements are built into the designs. Responsible organizations will ensure that the requirements
to adequately protect the environment are spelt out in terms of the scope and specifications of their projects, often more stringent than those required by the legislature. Mining, by its nature, depletes an asset i.e. as the asset (geological resource) is mined, the remaining asset is being depleted. As a result, other projects are required to be undertaken that will enhance sustainability such that when the mines’ reserves have been exploited, the community is not left out on a limb. Safety will always be a prime concern for the community. Many of the workers executing the project will be from the local community. The community does not want harm to come to any of their members, not even to mention the trauma that is felt by a seriously injured person’s family and friends. Safety issues are equally important to the client and to the project team. The measure of success in safetyrelated issues will therefore be similar for all three these stakeholders. The other people issues such as economic and social contribution, job creation, Black Economic Empowerment and affirmative action are issues that are addressed by the client in terms of the scope of the project and by the project management office’s policies and procedures.
PROJECT SUCCESS AND PERFORMACE EVALUATION
233
The work breakdown structure for the community’s requirements for a successful project will be as depicted in Figure 4. One can see that there is an exact overlap by the client’s indicators for scope and safety and by the project team’s factors for safety and team issues. A separate WBS is therefore not required in this instance because the community’s indicators are covered by the client’s and project team’s respective indices and WBS. Indices The WBS-like charts can be unpacked and weighted a further level for the client success indicators, as shown in Figure 5 below. Similarly, the team success factors can be developed a further level relating to process, compliance and efficacy. These factors can then be weighted, as shown in Figure 6. Project success evaluation Using these models, the success and performance of a project can be examined. A hypothetical example is shown in Table I. The absolute value of the rating is indicative of the client’s perception of the performance and success of the project, bearing in mind that only in a perfect world can an overall rating of 500 be achieved. The ratings in themselves also indicate that, in terms of cost and schedule, the performance of the project (and the team) was not good at all. As more and more projects are evaluated on this basis, project teams will be more sensitive to the client requirements of what constitutes a successful project and at the same time clients’ requirements and perceptions of project success will be more objective. Depending on the nature, type and size of the project, the breakdown of the indicators and factors can be taken down a further level, to level three. This evaluation can be done at different milestones of the project and during the different life cycles of the project.
Conclusion
The WBS method of evaluating project success, as proposed by Rad and Levin6, is an attempt to: • Reduce the subjectivity and perception issues such that the evaluation is more objective • Evaluate projects more consistently across different projects • Provide the project manager with a management tool that can assist him/her or her in ensuring greater project success. The author has attempted to modify and adapt the WBS in such a way that it will apply to projects in the South African context, taking into account the community’s requirements and including items such as safety, health and the environment. The shortcoming of this approach is that the weights of the different indices have only been estimated at this stage. Research will have to be conducted to determine and evaluate a project’s success. This then has to be compared with the results as obtained from the WBS evaluation of project success. The possible discrepancies between the research and the WBS approach can then be eliminated by adjusting the weights of the indices. The WBS evaluation will then give a very good and objective evaluation of project’s success. The author is currently in the process of conducting research in a project management organization in an attempt to match the project team’s and client’s views of project success to the evaluation results from a WBS approach.
References
1. Directorate: Mineral Economics. December 2003. South Africa’s Mineral Industry 2002/2003. Department of Minerals and Energy, South Africa pp. 14. 2. FREEMAN, M. and BEALE, P. Measuring Project Success. Project Management Journal, vol. XXIII, no. 1, 1992. pp. 8–17.
Things-related attributes
Scope as needed
Project success indicators
Safety, health and environment People-related attributes Team issues
Figure 4. Project success indicators: Community view
234
PLATINUM ADDING VALUE
Scope as needed 90
Original Final Justified variances Original
20 40 30 25 40 30 20 40 35 20 30 30 10 10 10 20 10 10 10 20 20 10 10
Quality as needed 95 Things-related attibutes 360 Schedule on time 95
Final Justified variances Original Final Justified variances Original
Project success indicators 500
Cost within budget 80
Final Justified variances Productivity
Team morale 50
Cooperation Absenteeism Demeanor
People-related attributes 140
Responsiveness Punctuality Client satisfaction 90 Innovation Trust Adversarial Listening Disclosing
Figure 5. Project success (weighted) indicators: client view5 (adapted)
3. Carroll, A.B. and Buchholtz, A.K. Business and Society: Ethics and Stakeholder Management. 4th ed. South-Western College College Publishing. United States of America. 2000. 4. MAYLOR, H. Project Management. 3rd ed. FT Prentice Hall, UK. 2003. 5. MEREDITH, J.R. and MANTEL, S.J. (Jr.). Project Management. A Managerial Approach. John Wily & Sons, Inc. United States of America. 2003. 6. RAD, P.F. and GINGER, L. The Advanced Project Management Office . A Comprehensive Look at Function and Implementation. Florida. St Lucie Press. 2002. 7. FREEMAN, M. and BEALE, P. Measuring Project Success. Project Management Journal, vol. XXII,
no. 4, 1991. pp. 23–27. 8. Project Management Institute. A Guide to the Project Management Body of Knowledge (PMBOK). Upper Darby: PMI. 2000. 9. PINTO, J.K. and SLEVIN, D.P. Project Success: Definitions and Measurement Techniques. Project Management Journal, vol. XIX, no. 1, 1998. p. 67. 10. SHENHAR, A.J., LEVY, O., and DVIR, D. Mapping the Dimensions of Project Success. Project Management Journal, vol. XXVIII, no. 3, 1997. p. 13. 11. GIDO, J. and CLEMENTS, J.P. 1999. Successful Project Management. Ohio. South-Western College Publishing. 12. READ, H.W. Project Performance Facets. Unpublished. 2003.
PROJECT SUCCESS AND PERFORMACE EVALUATION
235
Process Scope 60 Management of things issues 180 Quality 40 Efficacy
20 Cost 40
Process
13
Compliance 20 20
Compliance 13 Efficacy 14
Process
13 Schedule 40
Process
13
Compliance 13 Efficacy 14
Compliance 13 Efficacy 14
Process Integration 30
10
Compliance 10 Efficacy Process 10 10
Process Safety, health and environment 40
13
Compliance 13 Efficacy 14 Risk 30 Process 7
Project success factors 500
Management of things issues 140
Compliance 10 Efficacy Process 10 7 7 6
Reporting 20
Compliance 6 Efficacy 7 Contract admin. 20
Compliance Efficacy
Process Team 70 Management of things issues 180 Client 30
23 Suppliers & contractors 30
Process Compliance Efficacy
10 10 10
Compliance 23 Efficacy 24
Process
10 Communication 50
Process Compliance Efficacy
17 17 16
Compliance 10 Efficacy 10
Figure 6. Project success (weighted) factors: Team view5 (Adapted)
Table I Project Evaluation Example: English Channel Tunnel5
Client success indicators Scope Quality Cost Schedule Risk Integration Reporting Contract administration Communication Team Client Supplier Total/overall 75/95 75/95 20/90 30/80 Team success factors 40/60 30/40 20/40 30/40 8/40 10/30 20/30 15/40 40/50 20/70 20/30 20/30 273/500
15/50 30/90 245/500
236
PLATINUM ADDING VALUE
doc_704618900.pdf
Evaluation is a systematic determination of a subject's merit, worth and significance, using criteria governed by a set of standards. It can assist an organization, program, project or any other intervention or initiative to assess any aim, realisable concept/proposal, or any alternative, to help in decision-making; or to ascertain the degree of achievement or value in regard to the aim and objectives and results of any such action that has been completed.
KOELMANS, R.G. Project success and performance evaluation. International Platinum Conference ‘Platinum Adding Value’, The South African Institute of Mining and Metallurgy, 2004.
Project success and performance evaluation
R.G. KOELMANS
Read, Swatman & Voigt (Pty) Limited
In 2001 Platinum overtook gold in foreign revenue earnings for South Africa. The export sales from platinum group metals amounted to R29.3 billion as opposed to gold export sales of R28.6 billion1. Organizations must undertake new ventures to replace old, worn-out and obsolete assets with new ones. These assets could be a building, a new product or process, a new mine, or anything that can help the organization to achieve its objective. These ventures are projects by their nature. When a project is pronounced a success, the judgement is usually based on some factual evidence, although not every one uses the same data. From the literature, it is obvious that there is a lack of consensus and objectivity as to what constitutes a successful project. The purpose of this paper is to address the following questions: • What constitutes a successful project? • How does one measure project success? • How can one work toward project success? This paper addresses the requirements for project success of three of the primary stakeholders of a project, namely the business or client, the employees as far as the project team is concerned, and the local community. Indicators are developed into a work breakdown structure (WBS)-like chart to demonstrate success indicators. These indices take cognisance of all the constituents required for project success. The indicators are further developed to take cognisance of the requirements of the local communities. These indices are then weighted to arrive at a total score for the respective stakeholders. Using these WBS models, the success and performance of a project can be evaluated. This score is considered an objective measure of project success. Projects are evaluated more consistently across different projects and the project manager is provided with a management tool that can assist him/her in ensuring greater project success. The shortcoming of this approach is that the weights of the different indices have been estimated at this stage. Research will have to be conducted to more accurately reflect the weights assigned to the indices.
Introduction
Despite the significant development of project management principles in recent years, there are still a large proportion of projects that fail, or stated differently, projects that are not classed as being a success. Many projects have been and will still be undertaken in the platinum industry, be it expansion projects or process change projects. It makes sense then, unless some form of intervention occurs, that a large number of future projects could still be unsuccessful. Every one involved with a project wants it to be a success. Many cases can be cited from literature and anecdotal data of projects that fall short on expectations in one or more of the triple constraint items (time, cost and performance), or in terms of client satisfaction, and yet the project team officially pronounces the project a success. Success can mean different things to different people. An interesting example of this is provided: ‘An architect may consider success in terms of aesthetic appearance, an engineer in terms of technical competence, an accountant in
terms of dollars spent under budget, a human resources manager in terms of employee satisfaction. Chief executive officers rate their success in the stock market.’2 Freeman and Beale1 cite the fact that the motivation of their paper stems from the lack of consensus and objectivity as to what constitutes a successful project. This raises the following questions: • What constitutes a successful project? • How does one measure project success? • How can one work toward greater project success? The requirements of each stakeholder group will differ and therefore their perception of what constitutes success will vary. This paper analyses the requirements of three primary stakeholders and how these are measured. The requirements of some of these primary project stakeholders will initially be addressed, as well as project requirements, the goals of a project system, project constraints and what constitutes a successful project.
PROJECT SUCCESS AND PERFORMACE EVALUATION
229
Finally, this paper explores the development of a set of indices that can be used to evaluate project performance and success to make explicit what is implicit in seemingly subjective evaluations.
Project stakeholders
A stakeholder by definition is any one that has a stake in the project or its outcome. Stated differently, it is any interested party in the project or its outcome. Business stakeholders can be divided into primary and secondary and further subdivided into social and non-social stakeholders. Carroll and Buchholtz3 list the shareholders and investors (owners), employees and managers, customers, local communities, suppliers and other business partners as primary social stakeholders. When a business embarks on a project, a number of stakeholders can be identified and the list is by no means exhaustive: • The project sponsor and client • Owners and shareholders • Employees of the parent company undertaking the project • The community at large • The project team involved in the project • National and local governments • Suppliers who could be supplying products required by the project • The contractors who could benefit by working on the execution of the project • Competitors who could have a keen interest in the project as it may affect how they will operate in the future • Consumers. From these stakeholders, the paper will address the requirements for project success of some of the primary stakeholders, namely the business or client, the employees as far as the project team is concerned, and the local community.
There could be an even greater difference and variation in the goals of the community. In the South African context, the provision of additional jobs, social development and community involvement will rank high on the agenda of goals. Other objectives could be low noise and pollution levels. Increases in traffic volumes and noise could also be problematic to the community. Whether the scope of the project is for a product or constructed item, or a service, it will be translated into goals and objectives. These goals will be different for each stakeholder group but can be split into tangible goals such as time, cost, quality, and utility and intangible goals such as stakeholder satisfaction.
Constraints
A project can be viewed as an input-output model. Some form of input is transformed into output, under a set of constraints and utilizing a set of mechanisms to make the project happen3. The main constraints are time, cost and quality for the following reasons: • Time. Most if not all projects have a time constraint • Cost. Money to fund the project is a scarce resource. Clients want value for money • Quality. This is the standard by which the product, the end result of the project, and the process will be judged. In addition to the above main constraints, the following constraints can be limiting on the project4: • Legal issues have to be complied with • Ethical issues • Environmental issues • Logic—the need for certain activities to be completed before a project can start • Indirect effects—any change is most likely to have a ripple effect, which has to be taken into account. Trade-offs are invariably required to balance the constraints, especially the three main constraints. For example, a task may be completed very quickly through the addition of more resources, but unless one is prepared to compromise on quality, one will have to pay more. These trade-off decisions occur both during the planning and execution phase of the project4.
Project requirements
A project is started to fulfil a need or requirement. This requirement can be in the form of something tangible such as a new shaft system to access and exploit ore reserves, an improvement to an existing product such as a development waste loader, etc. or intangible such as developing a new mining method, a downsizing exercise, etc. A project is a temporary endeavour to create a unique product or service, has a start and end and is non-repetitive. These characteristics differentiate projects from operations4. The basic purpose of a project is to accomplish a specific goal5.
Measurement of project success
Projects are becoming more complex with the passage of time. There are a number of reasons why: • The simpler ideas have been exploited first and the need for innovation is vital • The allowable time to market is reducing, which impacts on one of the important constraints of all projects • Businesses are becoming more complex • Projects are also moving towards turnkey contracts, where the end user does not get involved with the interface between the parts of a system, but only deals with a single supplier in the provision of the entire system4. A vast number of articles and papers has been written and published on projects that have failed, sometimes with disastrous consequences. The explosion of the space shuttle, Challenger, just after take-off on 28 January 1986 and the more recent collapse of the Bushbuck Ridge bridge in Mpumalanga are examples of such major project failures. Many more can be cited.
Goals and objectives of a project system
The goals of each of the stakeholder groups will differ and vary according to the nature of the project. Generally, the client wants the required project with the required performance (quality and utility) for the lowest cost and shortest time. The scope of the project spells out what is to be done, by when and how much the costs will be. The project team goals will vary in urgency and sophistication. They could include to set industry standards, to integrate project management into the organization, to make a certain profit for the project management office, or to simply finish the project on time and within budget6.
230
PLATINUM ADDING VALUE
In market-driven economies organizations are established to supply goods or services to consumers. A private organization differs from a public one in that the former has a profit motive realized by exploiting competitive advantages in the market. Organizations must undertake new ventures to replace old, worn-out and obsolete assets with new ones. These assets could be a building, a new product or process, a new mine, or anything that can help the organization to achieve its objective7. These ventures are projects by their nature. A project by definition is a unique and temporary endeavour undertaken to create a product or service8. Anyone involved with a project wants it to be a success. The issue of project success is frequently discussed, yet ideas of what constitutes a successful project are many and varied. Pinto and Slevin9 cite that many people are aware of projects that come in on time and under budget and were nevertheless considered failures, yet the opposite is equally true. Similarly, Rad and Levin6 state that ‘many cases can be cited from the literature and anecdotal data of projects that fall short on expectations in one or more of the triple constraint items (time, cost and quality), or in terms of client satisfaction and yet the project team officially announces the project a success.’ Success can mean different things to different people. An interesting example of this has been provided: ‘An architect may consider success in terms of aesthetic appearance, an engineer in terms of technical competence, an accountant in terms of dollars spent under budget, a human resources manager in terms of employee satisfaction. Chief executive officers rate their success in the stock market.’2 Each project has different and unique stakeholders who have their own specific requirements for a project and therefore their own measure of what good project performance is and finally what a successful project entails. The further removed from the project, the more obscure these objectives and requirements may be. Different people, even if they are part of the same organization, will view success in a different light at different times10. According to Gido and Clements 11 , project success consists of four components namely budget (costs), schedule (time), performance (quality and utility), and customer satisfaction. The key to project success is the people, the project team and their organization (project management office), the tools and techniques used by the project team and the understanding the team has of the requirements and agendas of the stakeholders. One further aspect that has recently come to the fore is the question of the project’s impact on health, safety and the environment12. These combined criteria, factors and key issues can be grouped together as the constituents for project success and can be depicted as in Figure 1. Freeman and Beale1 cite the fact that the motivation for their paper stems from the lack of consensus and objectivity as to what constitutes a successful project. Project success is the most debated topic in the management field, but also the least agreed upon10. Discussion of project failures is never easy for the organizations concerned, but anecdotal data provide many examples of where a project falls short on one or more of the iron triangle items (time, cost and performance) or in terms of client satisfaction, but the team officially pronounce it a success. The converse has also occurred. The team may consider the project a success, while the client declares it a failure. The pronouncement of success or
failure may not even be unanimous among the team and or client personnel6. When a project is pronounced as a success, the judgment is usually based on some factual evidence, although not every one uses the same data. Even if different people use the same data, the same set of evaluation indices are not used in arriving at the degree of success of a project. The perception of success is sometimes based on feelings and personal indices5. The relationship between satisfaction, expectation and perception is quite simple. Satisfaction is determined by the difference between how the project is perceived or viewed by a stakeholder and how they expect the project to perform4. Satisfaction = perception - expectation From the above it is obvious that ‘thing-related’ issues such as data and ‘people-related’ issues such as perceptions play a role in whether a project is considered a success by the different stakeholders. How does one then measure project success? Success indicators and factors The three primary stakeholders in question will have different viewpoints of the success of a project. The client will be focused on the outcomes of the project, the deliverables. The focus will be on the goals and objectives of the project, more especially on the scope and quality aspects of the deliverables. In certain instances, time and costs issues are of secondary importance but could be just as important as the former aspects. The scope and quality of the project has been identified as the most influential factor among the elements of the iron triangle. The performance in cost and time (schedule) will be measured, similar to scope, against the final values and the relative magnitude of the variances. Certain variances can be justified, but the unjustified variance will influence the judgment whether the project was a success or not. These items lend themselves to quantification and therefore these indices lend themselves to mathematical precision. Independent of the above, the perception of success can be influenced by issues such as responsiveness of the team to client requests for changes, punctuality at meetings, communication or the lack thereof, personality clashes and
n tio ac isf at ts ien Cl
Utility
Sc he du le
Health, safety and environment
Quality
d an es ls iqu o n To ech t
Budge t
m tea ect e and ) j o Pr eopl ation (p aniz org
Figure 1. A schematic representation of the constituents of a successful project
PROJECT SUCCESS AND PERFORMACE EVALUATION
231
other people issues. These issues can be grouped into team morale and client satisfaction, and are harder to measure and quantify. The resulting quantity is usually subjective and can be open to lengthy debate6. Rad and Levin propose that these indicators be developed into a work breakdown structure (WBS)-like chart to demonstrate client success indicators 5 . A WBS is a breakdown of the project or work into smaller packages. The project is therefore subdivided into hierarchical units of tasks (level 1), subtasks (level 2), work packages (level 3) and so on. The WBS as proposed by Rad and Levin6 for both the client and the team takes cognizance of all the constituents as depicted in Figure 1 except for the safety, health and environmental issues. It can be argued that the requirements for these three issues form part of the scope, but the author has considered these items important enough to separate them. This WBS has been developed and unpacked for a project management and consulting engineering firm doing various projects for the platinum mines. This WBS is depicted in Figure 2. The project team, in turn, will be focused on the final deliverables and the process by which the deliverables are created. Included in the process are the tools and techniques used by the organization to execute its work. In order for a project to be considered a success by the team, all the activities indicated by the team factors must be managed well, regardless of the phase and of changes to scope, schedule and costs. The major difference between client success indicators and team success factors is that client indicators focus largely on the performance characteristics and external appearances of the product. The team success factors, on the other hand, focus on activities and measures that produce the deliverables of the project5. The project
team wants to deliver the project using best practices and procedures, which not only makes the team operate effectively, but also ensures a project that meets the required performance criteria and objectives of the client. In order to assess the success and effectiveness of the project manager and the team in the execution of their project, one has to identify three separate elements in the management process. The first is the existence of standardized procedures to do the work, the team’s conformity with these procedures, and finally how efficient these procedures make the work of the team. Project success factors can again be divided into two major categories similar to the client view, issues that deal with things and those issues that deal with people. In practice the things issues are tangible but intertwined with the people, the intangible issues. The degeneration of the people issues will have an impact on the things issues7. The next level of items that form the basis for project success factors are scope, cost, quality and schedule. According to Rad and Levin6, there are, however, items, which are things related but are also influenced by people to a certain degree. These are contract administration, project integration, reporting and risk. The management of these items requires documents, charts and reports (things) and also dealings with people, more so than scope, cost and quality. The management of the aforementioned items therefore contains an element of people issues. Level two of people issues include the management of the team issues, client issues, supplier and contractor issues and the various forms of communication. The relative weight placed on each of these issues will be influenced by the organization’s objectives, the type of project and project phase as shown in Figure 3.
Scope as needed
Quality as needed Things-related attributes Schedule on time Project success indicators
Cost within budget
Team morale People-related attibutes Client satisfaction
Figure 2. Project success indicators: client view5 (adapted)
232
PLATINUM ADDING VALUE
Scope Quality Things-related issues Schedule Cost Safety, health and environmental Integration Project success indicators Things-related issues Reporting Risk management Contract administration Team Suppliers & contractors Client Communication
People-related issues
Figure 3. Project success factors: team view5 (adapted)
For both the client and the project team, the aspects that determine a successful project, as viewed by the community, can be grouped into attributes related to things and those related to people. Issues that can be categorized as things would be the environmental impact and long-term sustainability of the project’s outcome. The people issues are considered to be safety, economic and social contribution, job creation, Black Economic Empowerment and affirmative action. Environmental issues are usually one of the first concerns of the community when a new project is launched. For this reason, legislation requires that an environmental impact assessment be carried out to ensure that mitigating actions are put in place, should there be a threat to the environment. Pollution, in all its forms (air, noise and water) is high on the community’s list of priorities. Energy consumption for example, can have a big impact on water consumption and air pollution through the generation of coal fired electricity. Legislation requires certain preventive measures to be in place to protect the environment. Often these requirements are barely sufficient as a minimum and when these projects are designed, such requirements are built into the designs. Responsible organizations will ensure that the requirements
to adequately protect the environment are spelt out in terms of the scope and specifications of their projects, often more stringent than those required by the legislature. Mining, by its nature, depletes an asset i.e. as the asset (geological resource) is mined, the remaining asset is being depleted. As a result, other projects are required to be undertaken that will enhance sustainability such that when the mines’ reserves have been exploited, the community is not left out on a limb. Safety will always be a prime concern for the community. Many of the workers executing the project will be from the local community. The community does not want harm to come to any of their members, not even to mention the trauma that is felt by a seriously injured person’s family and friends. Safety issues are equally important to the client and to the project team. The measure of success in safetyrelated issues will therefore be similar for all three these stakeholders. The other people issues such as economic and social contribution, job creation, Black Economic Empowerment and affirmative action are issues that are addressed by the client in terms of the scope of the project and by the project management office’s policies and procedures.
PROJECT SUCCESS AND PERFORMACE EVALUATION
233
The work breakdown structure for the community’s requirements for a successful project will be as depicted in Figure 4. One can see that there is an exact overlap by the client’s indicators for scope and safety and by the project team’s factors for safety and team issues. A separate WBS is therefore not required in this instance because the community’s indicators are covered by the client’s and project team’s respective indices and WBS. Indices The WBS-like charts can be unpacked and weighted a further level for the client success indicators, as shown in Figure 5 below. Similarly, the team success factors can be developed a further level relating to process, compliance and efficacy. These factors can then be weighted, as shown in Figure 6. Project success evaluation Using these models, the success and performance of a project can be examined. A hypothetical example is shown in Table I. The absolute value of the rating is indicative of the client’s perception of the performance and success of the project, bearing in mind that only in a perfect world can an overall rating of 500 be achieved. The ratings in themselves also indicate that, in terms of cost and schedule, the performance of the project (and the team) was not good at all. As more and more projects are evaluated on this basis, project teams will be more sensitive to the client requirements of what constitutes a successful project and at the same time clients’ requirements and perceptions of project success will be more objective. Depending on the nature, type and size of the project, the breakdown of the indicators and factors can be taken down a further level, to level three. This evaluation can be done at different milestones of the project and during the different life cycles of the project.
Conclusion
The WBS method of evaluating project success, as proposed by Rad and Levin6, is an attempt to: • Reduce the subjectivity and perception issues such that the evaluation is more objective • Evaluate projects more consistently across different projects • Provide the project manager with a management tool that can assist him/her or her in ensuring greater project success. The author has attempted to modify and adapt the WBS in such a way that it will apply to projects in the South African context, taking into account the community’s requirements and including items such as safety, health and the environment. The shortcoming of this approach is that the weights of the different indices have only been estimated at this stage. Research will have to be conducted to determine and evaluate a project’s success. This then has to be compared with the results as obtained from the WBS evaluation of project success. The possible discrepancies between the research and the WBS approach can then be eliminated by adjusting the weights of the indices. The WBS evaluation will then give a very good and objective evaluation of project’s success. The author is currently in the process of conducting research in a project management organization in an attempt to match the project team’s and client’s views of project success to the evaluation results from a WBS approach.
References
1. Directorate: Mineral Economics. December 2003. South Africa’s Mineral Industry 2002/2003. Department of Minerals and Energy, South Africa pp. 14. 2. FREEMAN, M. and BEALE, P. Measuring Project Success. Project Management Journal, vol. XXIII, no. 1, 1992. pp. 8–17.
Things-related attributes
Scope as needed
Project success indicators
Safety, health and environment People-related attributes Team issues
Figure 4. Project success indicators: Community view
234
PLATINUM ADDING VALUE
Scope as needed 90
Original Final Justified variances Original
20 40 30 25 40 30 20 40 35 20 30 30 10 10 10 20 10 10 10 20 20 10 10
Quality as needed 95 Things-related attibutes 360 Schedule on time 95
Final Justified variances Original Final Justified variances Original
Project success indicators 500
Cost within budget 80
Final Justified variances Productivity
Team morale 50
Cooperation Absenteeism Demeanor
People-related attributes 140
Responsiveness Punctuality Client satisfaction 90 Innovation Trust Adversarial Listening Disclosing
Figure 5. Project success (weighted) indicators: client view5 (adapted)
3. Carroll, A.B. and Buchholtz, A.K. Business and Society: Ethics and Stakeholder Management. 4th ed. South-Western College College Publishing. United States of America. 2000. 4. MAYLOR, H. Project Management. 3rd ed. FT Prentice Hall, UK. 2003. 5. MEREDITH, J.R. and MANTEL, S.J. (Jr.). Project Management. A Managerial Approach. John Wily & Sons, Inc. United States of America. 2003. 6. RAD, P.F. and GINGER, L. The Advanced Project Management Office . A Comprehensive Look at Function and Implementation. Florida. St Lucie Press. 2002. 7. FREEMAN, M. and BEALE, P. Measuring Project Success. Project Management Journal, vol. XXII,
no. 4, 1991. pp. 23–27. 8. Project Management Institute. A Guide to the Project Management Body of Knowledge (PMBOK). Upper Darby: PMI. 2000. 9. PINTO, J.K. and SLEVIN, D.P. Project Success: Definitions and Measurement Techniques. Project Management Journal, vol. XIX, no. 1, 1998. p. 67. 10. SHENHAR, A.J., LEVY, O., and DVIR, D. Mapping the Dimensions of Project Success. Project Management Journal, vol. XXVIII, no. 3, 1997. p. 13. 11. GIDO, J. and CLEMENTS, J.P. 1999. Successful Project Management. Ohio. South-Western College Publishing. 12. READ, H.W. Project Performance Facets. Unpublished. 2003.
PROJECT SUCCESS AND PERFORMACE EVALUATION
235
Process Scope 60 Management of things issues 180 Quality 40 Efficacy
20 Cost 40
Process
13
Compliance 20 20
Compliance 13 Efficacy 14
Process
13 Schedule 40
Process
13
Compliance 13 Efficacy 14
Compliance 13 Efficacy 14
Process Integration 30
10
Compliance 10 Efficacy Process 10 10
Process Safety, health and environment 40
13
Compliance 13 Efficacy 14 Risk 30 Process 7
Project success factors 500
Management of things issues 140
Compliance 10 Efficacy Process 10 7 7 6
Reporting 20
Compliance 6 Efficacy 7 Contract admin. 20
Compliance Efficacy
Process Team 70 Management of things issues 180 Client 30
23 Suppliers & contractors 30
Process Compliance Efficacy
10 10 10
Compliance 23 Efficacy 24
Process
10 Communication 50
Process Compliance Efficacy
17 17 16
Compliance 10 Efficacy 10
Figure 6. Project success (weighted) factors: Team view5 (Adapted)
Table I Project Evaluation Example: English Channel Tunnel5
Client success indicators Scope Quality Cost Schedule Risk Integration Reporting Contract administration Communication Team Client Supplier Total/overall 75/95 75/95 20/90 30/80 Team success factors 40/60 30/40 20/40 30/40 8/40 10/30 20/30 15/40 40/50 20/70 20/30 20/30 273/500
15/50 30/90 245/500
236
PLATINUM ADDING VALUE
doc_704618900.pdf