- Open Access
An experimental study on the impact of clinical interruptions on simulated trainee performances of central venous catheterization
Advances in Simulationvolume 2, Article number: 5 (2017)
Interruptions are common in the healthcare setting. This experimental study compares the effects of interruptions on simulated performances of central venous catheterization during a highly versus minimally complex portion of the task.
Twenty-six residents were assigned to interruptions during tasks that are (1) highly complex: establishing ultrasound-guided venous access (experimental group, n = 15) or (2) minimally complex: skin cleansing (control group, n = 11). Primary outcomes were (a) performance scores at three time points measured with a validated checklist, (b) time spent on the respective tasks, and (c) number of attempts to establish venous access.
Repeated measure analyses of variances of performance scores over time indicated no main effect of time or group. The interaction between time and group was significant: F (2, 44) = 4.28, p = 0.02, and partial eta2 = 0.16, indicating a large effect size. The experimental group scores decreased steadily over time, while the control group scores increased with time. The experimental group required longer to access the vein (148 s; interquartile range (IQR) 60 to 361 vs. 44 s; IQR 27 to 133 s; p = 0.034). Median number of attempts to establish venous access was higher in the experimental group (2, IQR 1–7 vs. 1, IQR 1–2; p = 0.03).
Interruptions during a highly complex task resulted in a consistent decrement in performance scores, longer time required to perform the task, and a higher number of venous access attempts than interruptions during a minimally complex tasks. We recommend avoiding interrupting trainees performing bedside procedures.
Interruptions in healthcare are common and occur ubiquitously. In an observational study of an intensive care unit, interruptions of healthcare professionals occurred at a rate of 14 times per hour , while physicians and nurses in the emergency room setting in a trauma center were observed to be interrupted more than ten times per hour . In the operating room, a mean of 50 interruptions was noted per case,  and on the medical ward, interruptions were also frequently present [4,5,6].
Given the limited working memory of individuals [7, 8], the impact of interruptions is such that, once interrupted, individuals may forget to resume the original task , take longer to complete the task , or complete tasks with higher error rates [11,12,13]. Overall, the yearly cost of interruptions to the hospital has been estimated to be more than US$51,000 per hospital . With over 5600 hospitals in the USA , interruptions are estimated to contribute to costs of over US$280 million per year.
Central venous catheter (CVC) insertion is a commonly performed procedure. In the USA, an estimated 20.1 million central-line days per year occur on inpatient wards . Although indicated for many medically ill patients, CVCs have associated complication risks, with an estimated complication rate of over 15% . The insertion of CVCs is a challenge for many trainees, as the procedure involves multiple steps [18, 19]. Further, while the use of ultrasound guidance is intended to improve patient safety [20, 21], its use adds to the complexity of the procedure.
Complex tasks have previously been shown to be more susceptible to the effects of interruptions than simpler tasks , and less experienced trainees are more susceptible to interruptions than experienced physicians . Given that many trainees at teaching centers perform ultrasound-guided CVC insertions , we hypothesize that trainees may be quite susceptible to interruptions and that disruption of their attentional focus may compromise procedural performances, especially during critical aspects of the task. As such, this study, via the use of simulation, seeks to examine the impact of interruptions at two different time points in the procedure performed by medical trainees and evaluates the impact of these interruptions on procedural performances. Specifically, we hypothesize that interruptions during a task that is complex, compared to interruptions during a task that is low in complexity, will result a lower procedural performance score, longer time spent on the procedural task, and a higher number of attempts to establish venous access.
All internal medicine residents (n = 98; postgraduate year (PGY)-1 to PGY-5) were invited to participate in this study between December 2012 and October 2013.
At baseline, all consenting participants completed a demographic survey. Participants were then given standardized instructions to place a CVC into the right internal jugular (IJ) vein using ultrasound guidance (SonixTOUCH, BK Ultrasound©) on a simulator (Gen II Ultrasound Central Line Training Model, Blue Phantom™), in a standardized procedure room. The participants were informed that the patient had chronic kidney disease and no peripheral intravenous access. One confederate nursing assistant was in the procedure room and provided assistance as requested by the participants. During the procedure, participants communicated with the patient, whose voice was controlled by researchers in the adjacent control room behind a one-way mirror. The scenario was video recorded using four camera angles, capturing views of the room, procedure site, procedural tray, and ultrasound screen (Fig. 1). Postprocedure, all participants underwent a 30-min semi-structured interview on their strategies for dealing with interruptions.
Participants were assigned to two groups in this study. Due to accidental violations in the randomization procedure, the majority of the participants (85%) were not randomized but assigned using unconcealed alternating group assignment. In the control group, participants were interrupted during a task that was felt to be low in complexity: skin cleaning for the insertion site. In the experimental group, participants were interrupted during a more complex task: establishing venous access under direct ultrasound guidance, where the interruption occurred as soon as the venous access needle entered the simulated skin.
At the pre-defined task (i.e., at the time of skin cleaning for the control group and at the time of venous needle skin entry for the experimental group), a 5-s interruption was introduced by a telephone call into the procedure room, whereby the nursing assistant relayed the message on the patient’s high potassium (7.9 mmol/L). An electrocardiogram indicating clinical severity (e.g., peaked T waves and widened QRS) was available if requested by the participant. The nursing assistant was instructed to acknowledge all orders from the participants except for intravenous orders, whereby the participants were reminded that the patient had no intravenous access. As therapy for the hyperkalemia requires intravenous access, it is anticipated that the participants would need to complete the CVC task.
The primary outcomes of interest were (1) overall performance of CVC insertion, (2) time spent on the respective tasks, and (3) number of attempts to establish venous access. Secondary outcomes included results from the thematic analyses of the interviews.
Performance of CVC insertion
Performance of CVC insertion was assessed using a 23-item checklist, modified from a previously published tool with validity evidence, to ensure that the items were applicable to our current task [25, 26].
Items that were executed appropriately were given a score of two, items that were not completed were given a zero, while items that were completed inappropriately or suboptimally were given a score of one. From this checklist, four scores were generated, presented as a percentage:
Overall score: sum of checklist score.
Time 1 score: steps prior to and including cleaning.
Time 2 score: steps after cleaning until venous access establishment.
Time 3 score: remaining steps in the procedure.
All performances were rated by a faculty (IM) with over 10 years of prior experience in rating CVC performances and previously demonstrated high inter-rater reliability using a similar tool . Blinding of the rater to group assignment was not possible as the videos clearly indicate when the interruptions occurred.
Time spent on procedure
Cleaning time was defined as the time taken to clean the insertion site. Time required to access the IJ vein was defined as the time from first needle puncture until successful venous access, as indicated by the removal of the syringe for wire insertion.
Number of attempts
The number of attempts taken to establish venous access using the needle and syringe was recorded. Number of attempts was recorded independently by two researchers (IM and MW). Inter-rater reliability for this measure was high [intraclass correlation coefficient = 0.97, 95% confidence interval (CI) 0.93 to 0.99].
Group differences were compared and analyzed in an intention-to-treat basis using standard parametric and non-parametric techniques . Construct validity of the checklist was assessed by comparing performance scores of junior trainees (PGY 1–2) with senior trainees (PGY 3–5): 71.7 ± standard deviation (SD) 22.8 vs. 88.1 ± 5.9%, respectively; p = 0.028. Internal reliability of the checklist was assessed using Cronbach’s alpha (alpha = 0.88).
After testing for the assumption of sphericity (not violated, chi-square (2) = 0.89, p = 0.64, epsilon = 0.96), mixed repeated measure analyses of variances were conducted to assess for group differences on performance scores on the three time points. Partial eta squared values are reported as measures of effect sizes and interpreted as follows: <0.01 = small effect, <0.06 = medium effect, and >0.14 = large effect . Significant interaction between group and time was further explored using the Bonferroni adjustments.
All performances were recorded and time coded with Noldus Recorder and Observer XT, version 11.0 (Noldus Information Technology, Wageningen, the Netherlands). All analyses were performed using SAS 9.3 (SAS Institute Inc., Cary, NC) and PASW Statistics, version 18.0 (PASW, IBM Corporation, Somers NY).
Qualitative data analyses
Interview data was transcribed into NVivo, version 10 (QSR International, Burlington, MA). Thematic content analysis was performed independently by two researchers (IM, JJ) . Assigned codes were reviewed and coded several times to ensure the saturation of themes. Codes were then grouped together based on similarities and linkages to form broader categories. Agreement in coding was high (Kappa = 0.89; 95% CI 0.87 to 0.90) . Disagreement in coding was resolved by consensus.
Twenty-six participants completed the study protocol. Of these, 11 (42%) to the control group and 15 (58%) were assigned to the experimental group. There were no significant demographic differences between the two groups (Table 1).
Overall checklist scores did not differ between groups (control group 82.7 ± SD 8.7% vs. experimental group 72.6 ± 23.4%; p = 0.16). Scores for the three time points are shown in Fig. 2. There was no significant main effect of time or group assignment: F (2, 44) = 0.08, p = 0.92, partial eta2 = 0.004; F (1, 22) = 0.46, p = 0.50; partial eta2 = 0.021, respectively. However, the interaction between time and group assignment was significant: F (2, 44) = 4.28, p = 0.02, partial eta2 = 0.16, indicating a large effect size.
Post-hoc pairwise comparisons were not significant (mean score difference [experimental group score – control group score] at time 1: 10.9 ± SD 10.5%, p = 0.31; time 2: −1.71 ± 5.0%, p = 0.73; time 3: −22.8 ± 11.7%, p = 0.06).
Time spent on procedure
Mean cleaning time did not differ between the two groups (control group 43 ± 17 s vs. experimental group 37 ± 13 s; p = 0.33).
Median time required to access the IJ vein was significantly longer in the experimental group (148 s; interquartile range [IQR] 60 to 361 vs. 44 s; IQR 27 to 133 s; p = 0.034).
Number of attempts
The median number of attempts to establish venous access was significantly higher in the experimental group (2, IQR 1–7) than that of the control group (1, IQR 1–2), p = 0.03.
Impact of interruptions
Two participants in the experimental group (13%) punctured the carotid artery while none in the control group did so. In the experimental group, technical errors observed included: one participant failed to aspirate during needle advancement while dealing with the interruption. Before and after the interruption, this participant did not display this suboptimal needle advancement technique. One participant left the needle open to air while managing the interruption. These errors were not observed in the control group.
In the control group, two participants cleaned the same area twice with the same sponge during the interruption. Another participant missed cleaning the center of the target area while being interrupted. These suboptimal cleaning techniques were not displayed during the first cleaning attempt before the interruption, nor were these techniques observed in the experimental group.
In the interview, participants reported being interrupted, outside of this study, a median of once per procedure (IQR 0 to 2). Analyses from semi-structured interviews revealed that the participants reported using an average of 4 ± SD 2 strategies in managing interruptions (Table 2). Task prioritization was the most commonly reported strategy (n = 19; 73%).
Our study identified that although performance scores do not differ between groups, interruptions during the experimental condition resulted in a number of serious procedural errors that were not observed in the control group. Examples of these errors included carotid puncture, leaving the needle open to air, and failure to aspirate during needle insertion. Further, a number of errors in cleaning technique were observed in the control group. These errors were likewise not observed in the experimental group, nor were they observed in the control group prior to the interruption.
Prior studies have examined the impact of interruptions on tasks such as peg or object transfer tasks and artificial distraction tasks, such as performing arithmetic [10,32,, 31–33]. In these studies, distractions typically were shown to result in a decrease in the performance of the distracting task , the primary task [10, 32], or both . However, there remains a need for further research on the impact of interruptions on clinical outcomes [34,35,36,37]. To our knowledge, our study is the first to explore the impact of interruptions using primary and interruption tasks specific to the CVC procedure, a complex procedure that is commonly performed . Our results identified technical errors as a result of interruptions. Further, the decrement in performance, time taken, and number of attempts made was significantly worse when the interruption occurred at a more complicated part of the procedure. Although our participants reported employing multiple strategies to manage the impact of interruptions, our results suggest that these strategies may be ineffective at preventing the negative consequences of interruptions. As such, limiting interruptions for trainee performances of CVC may be warranted .
Our study has a number of limitations. First, as a single-centered study, the generalizability of our conclusions may be limited. Our trainees were relatively inexperienced overall. Therefore, our results do not pertain to experienced proceduralists. Second, we did not assess the performance on the interrupting task itself. Potentially, participants whose CVC tasks suffered the most may have dealt with the interrupting task the best. However, since effective treatment for hyperkalemia required an intravenous access, and none of the participants chose to place an over-the-needle catheter into the IJ (all chose to complete the entire CVC insertion), performance on the interrupting task would have been immaterial. Third, although our group assignment was initially intended to be randomized, due to accidental violations in the randomization procedure, the majority of the participants were ultimately assigned in a non-randomized fashion (i.e., alternating group assignment). However, baseline participant characteristics were similar in both groups and no significant baseline differences were found in the two groups. Nonetheless, our study was not randomized in nature and therefore, we cannot exclude the possibility that the two groups systematically differed from each other. Fourth, our raters were not blinded to the group assignment and therefore are subject to potential bias. Further, we had duplicate raters only for some, not all, outcome measures. However, on those measures, our inter-rater reliability was high. Fifth, we did not perform a sample size calculation. We ultimately were only able to recruit a convenience sample of 26 participants, due to scheduling and availability issues and the voluntary nature of our study. Future studies should consider an a priori sample size calculation based on a single primary outcome. Sixth, we were unable to determine the exact time delay attributable to the cognitive effects of the interruptions, as we did not ask the participants to perform a think-aloud protocol. Future studies may consider the use of such a protocol. Seventh, we were unable to detect a difference in performance scores between groups, which may be a result of using a checklist which tended to award points for completing steps successfully . Potentially, the use of a checklist that specifically assesses for errors may be better suited to detect performance issues that arose as a result of the interruptions .
These limitations notwithstanding, overall, our study demonstrated that CVC performances are significantly impaired by interruptions, especially during a highly complex task. We therefore argue for the need to prevent these interruptions and/or develop effective strategies to assist trainees in mitigating the negative effects of interruptions on procedural performances. Previous studies have shown that visible signage and checklists may reduce the incidence of interruptions [37, 40]. In another study, the implementation of a “no interruption zone” for nurses resulted in a significant decrease in the number of interruptions . For CVC insertions, pager hand-off could be considered to help minimize interruptions . Nonetheless, some degree of interruption in healthcare may be unavoidable. As such, systems-wide strategies for reducing the impact of interruptions need to be examined, and educators should consider training learners to deal with procedural interruptions.
CVC performances are significantly impaired by interruptions. We recommend that trainees performing CVC insertions should not be interrupted during the procedure, especially during highly complex tasks.
Central venous catheter
Alvarez G, Coiera E. Interruptive communication patterns in the intensive care unit ward round. Int J Med Inform. 2005;74(10):791–6.
Brixey JJ, Tang Z, Robinson DJ, Johnson CW, Johnson TR, Turley JP, Patel VL, Zhang J. Interruptions in a level one trauma center: a case study. Int J Med Inform. 2008;77(4):235–41.
Healey AN, Sevdalis N, Vincent CA. Measuring intra-operative interference from distraction and interruption observed in the operating theatre. Ergonomics. 2006;49(5-6):589–604.
Lurie N, Rank B, Parenti C, Woolley T, Snoke W. How do house officers spend their nights? A time study of internal medicine house staff on call. N Engl J Med. 1989;320(25):1673–7.
Parenti C, Lurie N. Are things different in the light of day? A time study of internal medicine house staff days. Am J Med. 1993;94(6):654–8.
Ly T, Korb-Wells CS, Sumpton D, Russo RR, Barnsley L. Nature and impact of interruptions on clinical workflow of medical residents in the inpatient setting. J Grad Med Educ. 2013;5(2):232–7.
Baddeley A. The concept of working memory: a view of its current state and probable future development. Cognition. 1981;10(1):17–23.
Miller G. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev. 1956;63(2):81–97.
Westbrook JI, Coiera E, Dunsmuir WT, Brown BM, Kelk N, Paoloni R, Tran C. The impact of interruptions on clinical task completion. Qual Saf Health Care. 2010;19(4):284–9.
Goodell KH, Cao CG, Schwaitzberg SD. Effects of cognitive distraction on performance of laparoscopic surgical tasks. J Laparoendosc Adv S. 2006;16(2):94–8.
Flynn EA, Barker KN, Gibson JT, Pearson RE, Berger BA, Smith L. Impact of interruptions and distractions on dispensing errors in an ambulatory care pharmacy. Am J Health-Syst PH. 1999;56:1319–25.
Wiegmann DA, ElBardissi AW, Dearani JA, Daly RC, Sundt III TM. Disruptions in surgical flow and their relationship to surgical errors: an exploratory investigation. Surgery. 2007;142(5):658–65.
Elfering A, Grebner S, Ebener C. Workflow interruptions, cognitive failure and near-accidents in health care. Psychol Health Med. 2015;20(2):139–47.
Tucker AL. The impact of operational failures on hospital nurses and their patients. J Oper Manag. 2004;22(2):151–69.
American Hospital Association Fast Facts on US Hospital. 2016. http://www.aha.org/research/rc/stat-studies/fast-facts.shtml. Accessed 19 Jan 2017.
Raad I. Intravascular-catheter-related infections. Lancet. 1998;351(9106):893–8.
McGee DC, Gould MK. Preventing complications of central venous catheterization. N Engl J Med. 2003;348(12):1123–33.
Ma I, Sharma N, Brindle M, Caird J, McLaughlin K. Measuring competence in central venous catheterization: a systematic-review. Springer Plus. 2014;3(1):33.
Ortega R, Song M, Hansen CJ, Barash P. Ultrasound-guided internal jugular vein cannulation. N Engl J Med. 2010;362(16):e57.
National Institute for Health and Clinical Excellence (NICE). Technology Appraisal No 49: Guidance on the use of ultrasound locating devices for placing central venous catheters. 2002. https://www.nice.org.uk/guidance/TA49. Accessed 19 Jan 2017.
Feller-Kopman DMD. Ultrasound-guided central venous catheter placement: the new standard of care? Crit Care Med. 2005;33(8):1875–7.
Speier C, Vessey I, Valacich JS. The effects of interruptions, task complexity, and information presentation on computer-supported decision-making performance. Decision Sci. 2003;34(4):771–97.
Monteiro SD, Sherbino JD, Ilgen JS, Dore KL, Wood TJ, Young ME, Bandiera G, Blouin D, Gaissmaier W, Norman GR, et al. Disrupting diagnostic reasoning: do interruptions, instructions, and experience affect the diagnostic accuracy and response time of residents and emergency physicians? Acad Med. 2015;90(4):511–7.
Ma IW, Teteris E, Roberts JM, Bacchus M. Who is teaching and supervising our junior residents’ central venous catheterizations? BMC Med Educ. 2011;11(1):16.
Barsuk JH, McGaghie WC, Cohen ER, Balachandran JS, Wayne DB. Use of simulation-based mastery learning to improve the quality of central venous catheter placement in a medical intensive care unit. J Hosp Med. 2009;4(7):397–403.
Ma IW, Zalunardo N, Pachev G, Beran T, Brown M, Hatala R, McLaughlin K. Comparing the use of global rating scale with checklists for the assessment of central venous catheterization skills using simulation. Adv Health Sci Educ Theory Pract. 2012;17(4):457–70.
Walker GA, Shostak J. Common statistical methods for clinical research. 3rd ed. Cary: SAS Institute Inc; 2010.
Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. New York: Academic Press; 1988.
Fereday J, Muir-Cochrane E. Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. Int J Qual Methods. 2008;5(1):80–92.
Reichenheim ME. Confidence intervals for the kappa statistic. Stata J. 2004;4(4):421–8.
Park J, Waqar S, Kersey T, Modi N, Ong C, Sleep T. Effect of distraction on simulated anterior segment surgical performance. J Cataract Refr Surg. 2011;37(8):1517–22.
Moorthy K, Munz Y, Dosis A, Bann S, Darzi A. The effect of stress-inducing conditions on the performance of a laparoscopic task. Surg Endosc. 2003;17(9):1481–4.
Hsu K, Man FY, Gizicki R, Feldman L, Fried G. Experienced surgeons can do more than one thing at a time: effect of distraction on performance of a simple laparoscopic and cognitive task by experienced and novice surgeons. Surg Endosc. 2008;22(1):196–201.
Grundgeiger T, Sanderson P. Interruptions in healthcare: theoretical views. Int J Med Inform. 2009;78(5):293–307.
Coiera E. The science of interruption. BMJ Qual Saf. 2012;21(5):357–60.
Westbrook JI. Interruptions and multi-tasking: moving the research agenda in new directions. BMJ Qual Saf. 2014;23(11):877–9.
Rivera-Rodriguez AJ, Karsh B-T. Interruptions and distractions in healthcare: review and reappraisal. Qual Saf Health Care. 2010;19(4):304–12.
Walzak A, Bacchus M, Schaefer J, Zarnke K, Glow J, Brass C, McLaughlin K, Ma IWY. Diagnosing technical competence in six bedside procedures: comparing checklists and a global rating scale in the assessment of resident performance. Acad Med. 2015;90(8):1100–8.
Ma IWY, Pugh D, Mema B, Brindle ME, Cooke L, Stromer JN. Use of an error-focused checklist to identify incompetence in lumbar puncture performances. Med Educ. 2015;49(10):1004–15.
Pape TM, Guerra DM, Muzquiz M, Bryant JB, Ingram M, Schranner B, Alcala A, Sharp J, Bishop D, Carreno E, et al. Innovative approaches to reducing nurses’ distractions during medication administration. J Contin Educ Nure. 2005;36(3):108–16.
Anthony K, Wiencek C, Bauer C, Daly B, Anthony MK. No interruptions please: impact of a no interruption zone on medication safety in intensive care units. Crit Care Nurse. 2010;30(3):21–9.
Wall RJ, Ely EW, Elasy TA, Dittus RS, Foss J, Wilkerson KS, Speroff T. Using real time process measurements to reduce catheter related bloodstream infections in the intensive care unit. Qual Saf Health Care. 2005;14(4):295–302.
The authors wish to thank the residents who participated in this study, Greg Hallihan and Dr. William Ghali for their feedback on the study protocol, Dr. Craig Speizali for his help with participant recruitment, and Dr. Tak Fung for his assistance with the statistical analyses. Preliminary results from this study were presented in abstract form at Human Factors and Ergonomics Society 2014 International Symposium on Human Factors and Ergonomics in Health Care, March 16–19, 2014, Chicago, IL.
This work was supported by the W21C Research and Innovation Centre, Western Economic Diversification Fund, and the Faculty of Arts. The funding body has no role in the study design; data collection, analysis, and interpretation; the writing of the manuscript; or the decision to submit the manuscript for consideration for publication.
Availability of data and materials
The datasets generated and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request.
JJ contributed to the study design, data analysis and interpretation, and writing and critical revisions of the manuscript. MW contributed to the study design, data analysis and interpretation, and critical revisions of the manuscript. JC, AK, and AC contributed to the study design, data interpretation, and critical revisions of the manuscript. IWM contributed to the conception and design, data analysis and interpretation, and writing and critical revisions of the manuscript. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
The authors wish to thank Dr. Tara Cessford and Ms. Wrechelle Ocampo for their assistance with and consent for publication of the photographs.
Ethics approval and consent to participate
All participants included in this study provided written consent. This study was approved by the University of Calgary Conjoint Health Research Ethics Board (Ethics ID number: E-24877).