Description
Case Study on Risk Assessment May Not Always Lead To Successful Risk Control: A Systematic Literature Review of Risk Control after Root Cause Analysis:- The method by which firms evaluate potential losses and take action to reduce or eliminate such threats. Risk control is a technique that utilizes findings from risk assessments (identifying potential risk factors in a firm's operations, such as technical and non-technical aspects of the business, financial policies, and other policies that may impact the well-being of the firm), and implementing changes to reduce risk in these areas.
Case Study on Risk Assessment May Not Always Lead To Successful Risk Control: A Systematic Literature Review Of Risk Control after Root Cause Analysis
Abstract
Ro o t c a u s e a n a l y s i s i s p e r h a p s t h e m o s t w i d e l y u s e d t o o l i n h e a l t h c a r e r i s k m a n a g e m e n t, b ut d o e s it a c tua lly le a d to s uc c e s s ful r is k c o n tr o l? A r e th e r e c a te g o r ie s o f r is k c o n tr o l th a t a r e m o r e lik e ly to b e e ffe c tiv e ? A n d d o h e a lth c a r e r is k m a n a g e r s h a v e th e to o ls th e y n e e d to s up p o r t th e r is k c o n tr o l p r o c e s s ? T h is s y s te m a tic r e v ie w e x a m in e s h o w th e h e a lth c a r e s e c to r tr a n s la te s r is k a n a ly s is to r is k c o n tr o l a c tio n p la n s a n d e x a m in e s h o w to d o b e tte r . It s ug g e s ts th a t th e h ie r a r c h y o f r is k c o n tr o ls s h o uld in fo r m r is k c o n tr o l a c tio n p la n n in g a n d th a t n e w to o ls s h o uld b e d e v e lo p e d to im p r o v e th e r is k c o n tr o l p r o c e s s .
Introduction Root cause analysis (RCA) is a widely used approach to learning from patient safety incidents and near misses. In the US, its use is required by the Joint Commission (1) and the Veterans Health Administration, (2) and internationally it is mandated by governments as diverse as the UK, (3) Denmark, (4) and two states in Australia. (4) Although the term is familiar to healthcare risk managers across a variety of settings, the way RCA is implemented may vary considerably between and within organizations based on differing regulatory regimes, organizational cultures, or internal procedures. It is not a single, well-defined technique, but rather a general approach to uncovering the systems-level causes and contributing factors behind an incident or near-miss. As a risk analysis approach, its purpose is to inform risk evaluation (determining risk acceptability) and risk treatment (risk reduction). (5) As implemented in the Veterans? Health Administration, RCA has proven successful in its risk analysis
goals, increasing the number and quality of root causes identified. (6) But there is limited evidence to indicate that this improved risk analysis leads to patient safety improvement. (7,8) While risk matrices are commonly packaged with RCA to support risk evaluation, (9,10) it is less clear how the RCA process supports decisions about risk treatment (also known as risk control). We conducted a systematic literature review to determine what tools, if any, are being used to generate risk controls after an RCA in the healthcare sector, and whether certain categories of risk control (those higher up the hierarchy of risk controls) were associated with more successful outcomes. Methods Search strategy Databases: PubMed and EMBASE as of April 25th 2010. Search terms: "root cause analysis" OR "root-cause analysis" OR ("root cause" AND RCA) Exclusion criteria: Published before 2000, no application of RCA, not in healthcare, not in English, not original (repeats findings from another included article). Primary outcome measures Use of a systematic method for generating control recommendations, types of risk controls recommended and/or implemented, whether the controls proved successful. Risk Control Categorization Risk controls were categorized using a modified version of the National Institute for Occupational Safety and Health (NIOSH) hierarchy of risk controls. NIOSH (11) uses a five-tiered hierarchy: • Elimination
o
Stop using the hazardous process or materials Substitute a less hazardous process or material With a focus on physical barriers, isolation, forcing functions, human factors, and failsafe design, engineering controls improve safety "independent of worker interactions" (11)
• Substitution
o
• Engineering controls
o
?
• Administrative controls •
Policies, procedures, training and other controls that depend on people taking the correct actions Personal protective equipment (PPE)
o o
E.g. respirators, eye protection
For the purpose of this analysis, we have reduced this to three tiers: • Elimination o Same as above, but includes substitution as a mechanism for eliminating a hazard Design controls o Equivalent to engineering controls in the NIOSH hierarchy. Administrative controls o Same as above, but includes the use of PPE Wholesale substitution was subsumed by elimination, as it is simply a mechanism for achieving elimination. Personal protective equipment was rolled into the administrative controls category, because the use of PPE is dependent upon people taking the correct action. The term engineering control was changed to design control in order to make it less likely to inspire images of strictly mechanical control measures when used by healthcare workers. Secondary outcome measures Study demographics (e.g. country of origin), time and monetary costs associated with the RCA process, and how success was measured. Results Study Demographics Our search returned 231 unique articles, 60 of which met the inclusion criteria. Eighteen were excluded because they were published before 2000, fourteen because they did not pertain to healthcare, 127 because they did not describe an application of RCA, 11 because they were not written in English, and one because it simply repeated the findings of another included study. The included studies were overwhelmingly American and hospital-based. Not counting one study that took place in both countries, the US accounted for 41 of the studies, and the UK for six. No other country accounted for more than three. Nearly 80% (n=47) of the studies were from hospitals or hospital systems, not
?
• •
counting 5 nationwide studies from the US Veterans Health Administration (VHA). Risk Control Generation None of the studies reported the use of a systematic method for generating risk control recommendations. One, however, did report the use of force-field analysis for prioritizing potential risk controls after they had been generated. (12) Risk Controls Recommended and Implemented Twenty-eight studies reported a total of 196 risk controls recommended. Twentyeight studies (only 12 of which overlap with the first group) also reported a total of 123 risk controls implemented. Over 80% of risk control recommendations and 78% of implemented risk controls were administrative controls. Most of the remainder were design controls. Table 1 reports the mean/median number of each category of risk control per study, and the percentage of the risk controls represented by each category. Table 1. Risk Controls Recommended/Implemented by Category Risk Controls Recommended Mean* Median* Elimination Design Controls Administrative Controls 0.21 1.29 5.61 0.0 1.0 3.0 Percent** 1.5% 18.4% 80.1% Risk Controls Implemented Mean* Median* 0.14 0.82 3.43 0.0 0.0 3.0 Percent** 3.25% 18.7% 78.0%
* M e a n / m e d i a n n um b e r p e r s t ud y
**Percentage of pooled total from all studies (n = 28)
Implementation Rates Implementation rates, among the 12 studies that reported on both recommendations and implementation, were: Elimination, 50%; design controls, 65%; administrative controls, 78%. Success of Risk Controls Among the 28 studies that reported which risk controls had been implemented, half (n = 14) reported the use of administrative controls only. Those that described the use of elimination or design controls were 1.6 times more likely to have explicitly reported success than those that described the use of administrative controls alone. None of the studies reported that the risk controls had proven ineffective.
Measures of Success Of the 14 studies that explicitly stated that the risk controls had been successful, only two used any form of control to demonstrate that the apparent improvement was the result of the risk controls. Most described success in the form of reduced (or no additional) incidents of the type described, but it was not typically clear how this was measured. The use of audits or other active surveillance techniques was seldom noted. Time/Monetary Investment None of the studies reported the amount of money spent in relation to an RCA, and few reported the time investment. One reported 66 person-hours spent on an RCA related to a series of dispensing errors in community pharmacies. (13) Another required 640 person-hours from the core team alone, even though the "data needed for root cause analysis were collected before the first team meeting." (14) One suggested that "improvement teams should commit to a schedule of 90-minute weekly meetings for 7 weeks," but it was not clear whether the authors followed this advice, themselves, or how many team members were involved. (12) Mills, et al. of the US VHA reported that 143 single-incident RCAs related to adverse drug events (ADE) took a median of 35 person hours (range 6 - 1590 hours) (15) Mills, et al. also reported that the 176 aggregate RCAs examined in their paper on patient falls took 47.80 person hours, on average (SD = 32.40 hours) (16) and that 94 aggregate RCAs on suicide or suicidal behavior took 33.5 hours on average (SD = 25.8 hours). (17) Aggregate RCA is process through which a series of already completed RCAs on the same topic are assessed "?to identify trends and systems issues across groupings of similar events." (18) Mills et. al Although not included in the analysis of risk controls above (because the RCAs could not be tied to specific incidents, controls or outcomes), three aggregate RCA studies by Peter D. Mills and others from the VHA were very relevant to our examination of risk controls. In total, these covered 16,496 specific incidents, 65% related to patient falls, (19) 30% on ADE, (15) and the remaining 5% related to suicides. (17) A pooled total of 1,982 risk controls were slated for implementation as a response, 69.4% of which were fully implemented. The controls were categorized as focusing on: education/training (26.9%), policies and procedures (34.4%), and specific clinical changes to patient care (19.7%). The remaining controls (about 8% of the total) were characterized as "other." The first two categories fall under the heading of administrative controls, and amount to 61.3% of the total.
?
The true number is somewhat higher, however, as many of the interventions categorized as clinical changes also meet the definition of administrative controls. The papers on falls (19) and suicides (17) subcategorized the clinical change interventions, and on the basis of those subcategories, at least another 4.1% of the total would qualify as administrative controls. Thus, no less than 65.4% of all the risk controls reported in these studies fall into this lowest rung of the risk control hierarchy. Based on the 1,738 risk controls generated in the two largest studies, the authors found that risk controls based on training and education were negatively correlated with reports of improved outcomes, (15,19) that is, they made things worse. Actions focused on clinical changes and equipment/computers were found to be most effective. (15,19) Discussion After conducting an RCA, healthcare workers are left to their own devices in generating risk control plans. Implicit in this state of affairs is the notion that a good understanding of the risks will necessarily lead to good risk control. This may be a reasonable assumption in the industrial settings where RCA originated (where its use is often led by safety and reliability engineers), but healthcare workers are generally not trained in the principles of safety engineering. (8,20) They experience significant difficulty in generating and implementing risk treatment recommendations, and those they produce are often not consistent with best practice. (7,8,20-22) Indeed, some of the most popular risk control strategies in healthcare (training/education) may do more harm than good. (15,16) Perhaps as a result, there is limited evidence that RCA actually leads to patient safety improvement. (7,23) In the US, root cause analysis has been shown to increase the number, (6,8) and quality (6) of root causes / contributing factors identified. Introducing an analogous method to assist with the generation and assessment of robust risk controls might prove similarly successful, thus ensuring that this improved risk analysis does not go to waste. Pham, et al. have described an approach to selecting and implementing risk controls after they have been generated, (8) and one of the studies included in this review described the use of force-field analysis to prioritize potential risk controls. (12) But it does not appear that any widelyused methods exist for generating high-quality risk control options in the first place. Time/money invested and patient safety outcomes tend to be poorly reported, if at all, and strong publication bias is probably also at work, (7) making it impossible to draw any firm conclusions about the cost effectiveness of current
?
practice in RCA. However, the results of this study suggest that there may be scope for improvement. Conclusions Very little research has touched on risk control methods in healthcare. (24) This work provides evidence about the quality of risk control action planning after an RCA. It demonstrates that high-quality risk control plans do not reliably result from current practice, even among published RCAs, which are presumably of higher quality than run of the mill RCAs. "?Best practices have not been established for recommendations for action, follow-up, and analyzing results" after an RCA, (23) but administrative controls are widely accepted to be the weakest form of risk control. (11,15,16,21-25) James Bagian, one of the pioneers of healthcare RCA, has conducted an analysis of historical data from Joint Commission surveys that lends further support to this risk control hierarchy. He demonstrated that in scheduled surveys, there was relatively little difference in compliance rates between standards classified as administrative controls and those classified as engineering controls. But in unannounced surveys, hospitals were far less likely to be found in compliance with administrative controls vs. engineering controls. (26) The apparent implication: Engineering controls (i.e. design controls) work even when no one is watching; administrative controls often do not. Our comparison of studies that reported the exclusive use of administrative controls versus those that also employed elimination or design controls must be viewed as far from definitive, given the obvious publication bias among qualifying studies (i.e. none reported that the risk controls they had implemented proved unsuccessful), but our findings are consistent with the notion that administrative controls are generally less effective. The fact that we also demonstrated the healthcare industry?s overwhelming reliance on administrative controls is therefore quite troubling. In view of this, the use of formal methods to improve the quality of the risk treatment process may be warranted. References ? 1. The Joint Commission. I. Sentinel Events II. Goals of the Sentinel Event Policy III. Standards Relating to Sentinel Events Standards OrganizationSpecific Definition of Sentinel Event Expectations Under the Standards for an Organization?s Response to a Sentinel Event. The Joint Commission. 2007;:13. Bagian JP, Lee C, Gosbee J, DeRosier J, Stalhandske E, Eldridge N, et al. 2. Developing and deploying a patient safety program in a large health care
?
2.?
3. 4.
5. 6.
7.
8.
9.
10. 11. 12.
13.
14.
15.
16.
17.
delivery system: you can?t fix what you don't know about. Jt Comm J Qual Improv. 2001;27(10):522-532. NPSA. Root Cause Analysis (RCA) Toolkit [Internet]. 2004;Available from:http://www.msnpsa.nhs.uk/rcatoolkit/course/iindex.htm Taitz J, Genn K, Brooks V, Ross D, Ryan K, Shumack B, et al. Systemwide learning from root cause analysis: a report from the New South Wales Root Cause Analysis Review Committee. Quality & Safety in Health Care. 2010 Jul [cited 2010 Dec 2];19(6):e63. ISO. ISO 31000: Risk management — Principles and guidelines on implementation. 2008; Bagian JP, Gosbee J, Lee CZ, Williams L, McKnight SD, Mannos DM. The Veterans Affairs root cause analysis system in action. Jt Comm J Qual Improv. 2002;28(10):531-545. Percarpio KB, Watts BV, Weeks WB. The effectiveness of root cause analysis: what does the literature tell us? Joint Commission Journal on Quality & Safety. 2008 Jul;34(7):391-8. Pham JC, Kim GR, Natterman JP, Cover RM, Goeschel CA, Wu AW, et al. ReCASTing the RCA: an improved model for performing root cause analyses. American Journal of Medical Quality?: 2010 Jan;25(3):186-91. NPSA. A risk matrix for risk managers [Internet]. 2008;Available from:http://www.nrls.npsa.nhs.uk/EasySiteWeb/getresource.axd?AssetID=6014 9&type=full&servicetype=Attachment VHA. VHA National Patient Safety Improvement Handbook. 2008;:22. NIOSH. NIOSH Topic?: Engineering Controls | CDC NIOSH [Internet]. 2010;Available from:http://www.cdc.gov/niosh/topics/engcontrols/ Burroughs TE, Cira JC, Chartock P, Davies AR, Dunagan WC. Using root cause analysis to address patient satisfaction and other improvement opportunities. Jt Comm J Qual Improv. 2000;26(8):439-449. Knudsen P, Herborg H, Mortensen AR, Knudsen M, Hellebek A. Preventing medication errors in community pharmacy: root-cause analysis of transcription errors. Qual Saf Health Care. 2007;16(4):285-290. Hellwig SD, Piper L, Naylor E. Forty hours under pressure: a rapidresponse improvement team achieves synergy. J Healthc Qual. 2002;24(3):21-23,35. Mills PD, Neily J, Kinney LM, Bagian J, Weeks WB. Effective interventions and implementation strategies to reduce adverse drug events in the Veterans Affairs (VA) system. Quality & Safety in Health Care. 2008;17(1):37-46. Mills PD, Neily J, Luan D, Stalhandske E, Weeks W. Using Aggregate Root Cause Analysis to Reduce Falls and Related Injuries. Joint Commission Journal on Quality and Patient Safety. 2005;31(1):21-31. Mills PD, Neily J, Luan D, Osborne A, Howard K. Actions and Implementation Strategies to Reduce Suicidal Events in the Veterans
18.
19.
20. 21. 22.
23. 24.
25. 26.
Health Administration. Joint Commission Journal on Quality and Patient Safety. 2006;32(3):12. Neily J, Ogrinc G, Mills P, Williams R, Stalhandske E, Bagian J, et al. Using aggregate root cause analysis to improve patient safety. Jt Comm J Qual Saf. 2003 Aug;29(8):381,434-439. Mills PD, Neily J, Luan D, Stalhandske E, Weeks WB. Using aggregate root cause analysis to reduce falls. Jt Comm J Qual Patient Saf. 2005;31(1):21-31. Youngson GG, Flin R. Patient safety in surgery: non-technical aspects of safe surgical performance. Patient Safety in Surgery. 2010 Jan;4(1):4. Lyons M, Woloshynowych M, Adams S, Vincent C. Error reduction in medicine: Final report to the Nuffield Trust. Nuffield Trust; 2005. Wallace LM, Spurgeon P, Earll L. Evaluation of the NPSA 3 Day Root Cause Analysis Training Programme?: Final Report (Revised October 2006). 2006. Wu AW, Lipshutz AKM, Pronovost PJ. Effectiveness and efficiency of root cause analysis in medicine. JAMA?. 2008;299(6):685-7. Dückers M, Faber M, Cruijsberg J, Grol R, Schoonhoven L, Wensing M. Safety and risk management interventions in hospitals: a systematic review of the literature. Medical care research and review?: MCRR. 2009 Dec;66(6 Suppl):90S-119S. Manuele FA. Risk assessment and hierarchies of control. Professional Safety. 2005;50(5):33-39. Bagian. Personal communication. 2011;
doc_582340337.docx
Case Study on Risk Assessment May Not Always Lead To Successful Risk Control: A Systematic Literature Review of Risk Control after Root Cause Analysis:- The method by which firms evaluate potential losses and take action to reduce or eliminate such threats. Risk control is a technique that utilizes findings from risk assessments (identifying potential risk factors in a firm's operations, such as technical and non-technical aspects of the business, financial policies, and other policies that may impact the well-being of the firm), and implementing changes to reduce risk in these areas.
Case Study on Risk Assessment May Not Always Lead To Successful Risk Control: A Systematic Literature Review Of Risk Control after Root Cause Analysis
Abstract
Ro o t c a u s e a n a l y s i s i s p e r h a p s t h e m o s t w i d e l y u s e d t o o l i n h e a l t h c a r e r i s k m a n a g e m e n t, b ut d o e s it a c tua lly le a d to s uc c e s s ful r is k c o n tr o l? A r e th e r e c a te g o r ie s o f r is k c o n tr o l th a t a r e m o r e lik e ly to b e e ffe c tiv e ? A n d d o h e a lth c a r e r is k m a n a g e r s h a v e th e to o ls th e y n e e d to s up p o r t th e r is k c o n tr o l p r o c e s s ? T h is s y s te m a tic r e v ie w e x a m in e s h o w th e h e a lth c a r e s e c to r tr a n s la te s r is k a n a ly s is to r is k c o n tr o l a c tio n p la n s a n d e x a m in e s h o w to d o b e tte r . It s ug g e s ts th a t th e h ie r a r c h y o f r is k c o n tr o ls s h o uld in fo r m r is k c o n tr o l a c tio n p la n n in g a n d th a t n e w to o ls s h o uld b e d e v e lo p e d to im p r o v e th e r is k c o n tr o l p r o c e s s .
Introduction Root cause analysis (RCA) is a widely used approach to learning from patient safety incidents and near misses. In the US, its use is required by the Joint Commission (1) and the Veterans Health Administration, (2) and internationally it is mandated by governments as diverse as the UK, (3) Denmark, (4) and two states in Australia. (4) Although the term is familiar to healthcare risk managers across a variety of settings, the way RCA is implemented may vary considerably between and within organizations based on differing regulatory regimes, organizational cultures, or internal procedures. It is not a single, well-defined technique, but rather a general approach to uncovering the systems-level causes and contributing factors behind an incident or near-miss. As a risk analysis approach, its purpose is to inform risk evaluation (determining risk acceptability) and risk treatment (risk reduction). (5) As implemented in the Veterans? Health Administration, RCA has proven successful in its risk analysis
goals, increasing the number and quality of root causes identified. (6) But there is limited evidence to indicate that this improved risk analysis leads to patient safety improvement. (7,8) While risk matrices are commonly packaged with RCA to support risk evaluation, (9,10) it is less clear how the RCA process supports decisions about risk treatment (also known as risk control). We conducted a systematic literature review to determine what tools, if any, are being used to generate risk controls after an RCA in the healthcare sector, and whether certain categories of risk control (those higher up the hierarchy of risk controls) were associated with more successful outcomes. Methods Search strategy Databases: PubMed and EMBASE as of April 25th 2010. Search terms: "root cause analysis" OR "root-cause analysis" OR ("root cause" AND RCA) Exclusion criteria: Published before 2000, no application of RCA, not in healthcare, not in English, not original (repeats findings from another included article). Primary outcome measures Use of a systematic method for generating control recommendations, types of risk controls recommended and/or implemented, whether the controls proved successful. Risk Control Categorization Risk controls were categorized using a modified version of the National Institute for Occupational Safety and Health (NIOSH) hierarchy of risk controls. NIOSH (11) uses a five-tiered hierarchy: • Elimination
o
Stop using the hazardous process or materials Substitute a less hazardous process or material With a focus on physical barriers, isolation, forcing functions, human factors, and failsafe design, engineering controls improve safety "independent of worker interactions" (11)
• Substitution
o
• Engineering controls
o
?
• Administrative controls •
Policies, procedures, training and other controls that depend on people taking the correct actions Personal protective equipment (PPE)
o o
E.g. respirators, eye protection
For the purpose of this analysis, we have reduced this to three tiers: • Elimination o Same as above, but includes substitution as a mechanism for eliminating a hazard Design controls o Equivalent to engineering controls in the NIOSH hierarchy. Administrative controls o Same as above, but includes the use of PPE Wholesale substitution was subsumed by elimination, as it is simply a mechanism for achieving elimination. Personal protective equipment was rolled into the administrative controls category, because the use of PPE is dependent upon people taking the correct action. The term engineering control was changed to design control in order to make it less likely to inspire images of strictly mechanical control measures when used by healthcare workers. Secondary outcome measures Study demographics (e.g. country of origin), time and monetary costs associated with the RCA process, and how success was measured. Results Study Demographics Our search returned 231 unique articles, 60 of which met the inclusion criteria. Eighteen were excluded because they were published before 2000, fourteen because they did not pertain to healthcare, 127 because they did not describe an application of RCA, 11 because they were not written in English, and one because it simply repeated the findings of another included study. The included studies were overwhelmingly American and hospital-based. Not counting one study that took place in both countries, the US accounted for 41 of the studies, and the UK for six. No other country accounted for more than three. Nearly 80% (n=47) of the studies were from hospitals or hospital systems, not
?
• •
counting 5 nationwide studies from the US Veterans Health Administration (VHA). Risk Control Generation None of the studies reported the use of a systematic method for generating risk control recommendations. One, however, did report the use of force-field analysis for prioritizing potential risk controls after they had been generated. (12) Risk Controls Recommended and Implemented Twenty-eight studies reported a total of 196 risk controls recommended. Twentyeight studies (only 12 of which overlap with the first group) also reported a total of 123 risk controls implemented. Over 80% of risk control recommendations and 78% of implemented risk controls were administrative controls. Most of the remainder were design controls. Table 1 reports the mean/median number of each category of risk control per study, and the percentage of the risk controls represented by each category. Table 1. Risk Controls Recommended/Implemented by Category Risk Controls Recommended Mean* Median* Elimination Design Controls Administrative Controls 0.21 1.29 5.61 0.0 1.0 3.0 Percent** 1.5% 18.4% 80.1% Risk Controls Implemented Mean* Median* 0.14 0.82 3.43 0.0 0.0 3.0 Percent** 3.25% 18.7% 78.0%
* M e a n / m e d i a n n um b e r p e r s t ud y
**Percentage of pooled total from all studies (n = 28)
Implementation Rates Implementation rates, among the 12 studies that reported on both recommendations and implementation, were: Elimination, 50%; design controls, 65%; administrative controls, 78%. Success of Risk Controls Among the 28 studies that reported which risk controls had been implemented, half (n = 14) reported the use of administrative controls only. Those that described the use of elimination or design controls were 1.6 times more likely to have explicitly reported success than those that described the use of administrative controls alone. None of the studies reported that the risk controls had proven ineffective.
Measures of Success Of the 14 studies that explicitly stated that the risk controls had been successful, only two used any form of control to demonstrate that the apparent improvement was the result of the risk controls. Most described success in the form of reduced (or no additional) incidents of the type described, but it was not typically clear how this was measured. The use of audits or other active surveillance techniques was seldom noted. Time/Monetary Investment None of the studies reported the amount of money spent in relation to an RCA, and few reported the time investment. One reported 66 person-hours spent on an RCA related to a series of dispensing errors in community pharmacies. (13) Another required 640 person-hours from the core team alone, even though the "data needed for root cause analysis were collected before the first team meeting." (14) One suggested that "improvement teams should commit to a schedule of 90-minute weekly meetings for 7 weeks," but it was not clear whether the authors followed this advice, themselves, or how many team members were involved. (12) Mills, et al. of the US VHA reported that 143 single-incident RCAs related to adverse drug events (ADE) took a median of 35 person hours (range 6 - 1590 hours) (15) Mills, et al. also reported that the 176 aggregate RCAs examined in their paper on patient falls took 47.80 person hours, on average (SD = 32.40 hours) (16) and that 94 aggregate RCAs on suicide or suicidal behavior took 33.5 hours on average (SD = 25.8 hours). (17) Aggregate RCA is process through which a series of already completed RCAs on the same topic are assessed "?to identify trends and systems issues across groupings of similar events." (18) Mills et. al Although not included in the analysis of risk controls above (because the RCAs could not be tied to specific incidents, controls or outcomes), three aggregate RCA studies by Peter D. Mills and others from the VHA were very relevant to our examination of risk controls. In total, these covered 16,496 specific incidents, 65% related to patient falls, (19) 30% on ADE, (15) and the remaining 5% related to suicides. (17) A pooled total of 1,982 risk controls were slated for implementation as a response, 69.4% of which were fully implemented. The controls were categorized as focusing on: education/training (26.9%), policies and procedures (34.4%), and specific clinical changes to patient care (19.7%). The remaining controls (about 8% of the total) were characterized as "other." The first two categories fall under the heading of administrative controls, and amount to 61.3% of the total.
?
The true number is somewhat higher, however, as many of the interventions categorized as clinical changes also meet the definition of administrative controls. The papers on falls (19) and suicides (17) subcategorized the clinical change interventions, and on the basis of those subcategories, at least another 4.1% of the total would qualify as administrative controls. Thus, no less than 65.4% of all the risk controls reported in these studies fall into this lowest rung of the risk control hierarchy. Based on the 1,738 risk controls generated in the two largest studies, the authors found that risk controls based on training and education were negatively correlated with reports of improved outcomes, (15,19) that is, they made things worse. Actions focused on clinical changes and equipment/computers were found to be most effective. (15,19) Discussion After conducting an RCA, healthcare workers are left to their own devices in generating risk control plans. Implicit in this state of affairs is the notion that a good understanding of the risks will necessarily lead to good risk control. This may be a reasonable assumption in the industrial settings where RCA originated (where its use is often led by safety and reliability engineers), but healthcare workers are generally not trained in the principles of safety engineering. (8,20) They experience significant difficulty in generating and implementing risk treatment recommendations, and those they produce are often not consistent with best practice. (7,8,20-22) Indeed, some of the most popular risk control strategies in healthcare (training/education) may do more harm than good. (15,16) Perhaps as a result, there is limited evidence that RCA actually leads to patient safety improvement. (7,23) In the US, root cause analysis has been shown to increase the number, (6,8) and quality (6) of root causes / contributing factors identified. Introducing an analogous method to assist with the generation and assessment of robust risk controls might prove similarly successful, thus ensuring that this improved risk analysis does not go to waste. Pham, et al. have described an approach to selecting and implementing risk controls after they have been generated, (8) and one of the studies included in this review described the use of force-field analysis to prioritize potential risk controls. (12) But it does not appear that any widelyused methods exist for generating high-quality risk control options in the first place. Time/money invested and patient safety outcomes tend to be poorly reported, if at all, and strong publication bias is probably also at work, (7) making it impossible to draw any firm conclusions about the cost effectiveness of current
?
practice in RCA. However, the results of this study suggest that there may be scope for improvement. Conclusions Very little research has touched on risk control methods in healthcare. (24) This work provides evidence about the quality of risk control action planning after an RCA. It demonstrates that high-quality risk control plans do not reliably result from current practice, even among published RCAs, which are presumably of higher quality than run of the mill RCAs. "?Best practices have not been established for recommendations for action, follow-up, and analyzing results" after an RCA, (23) but administrative controls are widely accepted to be the weakest form of risk control. (11,15,16,21-25) James Bagian, one of the pioneers of healthcare RCA, has conducted an analysis of historical data from Joint Commission surveys that lends further support to this risk control hierarchy. He demonstrated that in scheduled surveys, there was relatively little difference in compliance rates between standards classified as administrative controls and those classified as engineering controls. But in unannounced surveys, hospitals were far less likely to be found in compliance with administrative controls vs. engineering controls. (26) The apparent implication: Engineering controls (i.e. design controls) work even when no one is watching; administrative controls often do not. Our comparison of studies that reported the exclusive use of administrative controls versus those that also employed elimination or design controls must be viewed as far from definitive, given the obvious publication bias among qualifying studies (i.e. none reported that the risk controls they had implemented proved unsuccessful), but our findings are consistent with the notion that administrative controls are generally less effective. The fact that we also demonstrated the healthcare industry?s overwhelming reliance on administrative controls is therefore quite troubling. In view of this, the use of formal methods to improve the quality of the risk treatment process may be warranted. References ? 1. The Joint Commission. I. Sentinel Events II. Goals of the Sentinel Event Policy III. Standards Relating to Sentinel Events Standards OrganizationSpecific Definition of Sentinel Event Expectations Under the Standards for an Organization?s Response to a Sentinel Event. The Joint Commission. 2007;:13. Bagian JP, Lee C, Gosbee J, DeRosier J, Stalhandske E, Eldridge N, et al. 2. Developing and deploying a patient safety program in a large health care
?
2.?
3. 4.
5. 6.
7.
8.
9.
10. 11. 12.
13.
14.
15.
16.
17.
delivery system: you can?t fix what you don't know about. Jt Comm J Qual Improv. 2001;27(10):522-532. NPSA. Root Cause Analysis (RCA) Toolkit [Internet]. 2004;Available from:http://www.msnpsa.nhs.uk/rcatoolkit/course/iindex.htm Taitz J, Genn K, Brooks V, Ross D, Ryan K, Shumack B, et al. Systemwide learning from root cause analysis: a report from the New South Wales Root Cause Analysis Review Committee. Quality & Safety in Health Care. 2010 Jul [cited 2010 Dec 2];19(6):e63. ISO. ISO 31000: Risk management — Principles and guidelines on implementation. 2008; Bagian JP, Gosbee J, Lee CZ, Williams L, McKnight SD, Mannos DM. The Veterans Affairs root cause analysis system in action. Jt Comm J Qual Improv. 2002;28(10):531-545. Percarpio KB, Watts BV, Weeks WB. The effectiveness of root cause analysis: what does the literature tell us? Joint Commission Journal on Quality & Safety. 2008 Jul;34(7):391-8. Pham JC, Kim GR, Natterman JP, Cover RM, Goeschel CA, Wu AW, et al. ReCASTing the RCA: an improved model for performing root cause analyses. American Journal of Medical Quality?: 2010 Jan;25(3):186-91. NPSA. A risk matrix for risk managers [Internet]. 2008;Available from:http://www.nrls.npsa.nhs.uk/EasySiteWeb/getresource.axd?AssetID=6014 9&type=full&servicetype=Attachment VHA. VHA National Patient Safety Improvement Handbook. 2008;:22. NIOSH. NIOSH Topic?: Engineering Controls | CDC NIOSH [Internet]. 2010;Available from:http://www.cdc.gov/niosh/topics/engcontrols/ Burroughs TE, Cira JC, Chartock P, Davies AR, Dunagan WC. Using root cause analysis to address patient satisfaction and other improvement opportunities. Jt Comm J Qual Improv. 2000;26(8):439-449. Knudsen P, Herborg H, Mortensen AR, Knudsen M, Hellebek A. Preventing medication errors in community pharmacy: root-cause analysis of transcription errors. Qual Saf Health Care. 2007;16(4):285-290. Hellwig SD, Piper L, Naylor E. Forty hours under pressure: a rapidresponse improvement team achieves synergy. J Healthc Qual. 2002;24(3):21-23,35. Mills PD, Neily J, Kinney LM, Bagian J, Weeks WB. Effective interventions and implementation strategies to reduce adverse drug events in the Veterans Affairs (VA) system. Quality & Safety in Health Care. 2008;17(1):37-46. Mills PD, Neily J, Luan D, Stalhandske E, Weeks W. Using Aggregate Root Cause Analysis to Reduce Falls and Related Injuries. Joint Commission Journal on Quality and Patient Safety. 2005;31(1):21-31. Mills PD, Neily J, Luan D, Osborne A, Howard K. Actions and Implementation Strategies to Reduce Suicidal Events in the Veterans
18.
19.
20. 21. 22.
23. 24.
25. 26.
Health Administration. Joint Commission Journal on Quality and Patient Safety. 2006;32(3):12. Neily J, Ogrinc G, Mills P, Williams R, Stalhandske E, Bagian J, et al. Using aggregate root cause analysis to improve patient safety. Jt Comm J Qual Saf. 2003 Aug;29(8):381,434-439. Mills PD, Neily J, Luan D, Stalhandske E, Weeks WB. Using aggregate root cause analysis to reduce falls. Jt Comm J Qual Patient Saf. 2005;31(1):21-31. Youngson GG, Flin R. Patient safety in surgery: non-technical aspects of safe surgical performance. Patient Safety in Surgery. 2010 Jan;4(1):4. Lyons M, Woloshynowych M, Adams S, Vincent C. Error reduction in medicine: Final report to the Nuffield Trust. Nuffield Trust; 2005. Wallace LM, Spurgeon P, Earll L. Evaluation of the NPSA 3 Day Root Cause Analysis Training Programme?: Final Report (Revised October 2006). 2006. Wu AW, Lipshutz AKM, Pronovost PJ. Effectiveness and efficiency of root cause analysis in medicine. JAMA?. 2008;299(6):685-7. Dückers M, Faber M, Cruijsberg J, Grol R, Schoonhoven L, Wensing M. Safety and risk management interventions in hospitals: a systematic review of the literature. Medical care research and review?: MCRR. 2009 Dec;66(6 Suppl):90S-119S. Manuele FA. Risk assessment and hierarchies of control. Professional Safety. 2005;50(5):33-39. Bagian. Personal communication. 2011;
doc_582340337.docx