Skip to main content

Table 2 Considerations for the “Process phase” of translational simulation. Examples of tools and techniques for data collection and analysis*

From: Translational simulation: from description to action

Direct observation Observers
selected for expertise; may need training in assessment tools
Assessment tools
Procedure-specific assessment tools, e.g. arterial blood sampling [37]
Global rating scale for procedural skills [37]
Teamwork, e.g. Team Emergency Assessment Measure (TEAM) [38]
Time-to-event, e.g. time to CT scan for trauma patients [8]
Safe design goals observer tool [39]
Ethnographic observation [6]
Monitoring Video and/or audio recording and streaming
Motion tracking [5]
Eye movement tracking [40]
Other ergonomic assessment tools (e.g. heart rate monitoring, strain measurements) [41]
Learning conversations Debrief approaches
Rapid cycle deliberate practice [42] (can be modified to improve processes as well as individual performance)
Systems-focused Promoting Excellence and Reflective Learning in Simulation (PEARLS) framework [43]
SAFEE debriefing tool [44] (based on evidence-based design principles)
Pluralist walkthrough [41] with iterative discussions
‘Brainstorm’ sessions [45, 46] (e.g. with participants having the opportunity for quiet reflection and labelling the environment and equipment with sticky notes as a starting point for discussion)
Documentation
Whiteboards, sticky notes, and photography
Technology-enhanced (e.g. TrelloTM [47] as a virtual ‘sticky note board’)
Video recording, audio recording and transcription
Post-event data Review of video recording, audio recording and transcripts
Interviews and focus groups [45, 46]
Surveys, e.g., Relational Coordination Survey [6]
Artefact analysis (e.g. guidelines, cognitive aids, checklists, debrief reports) [41]
Analysis General [42, 44, 45]
Qualitative analysis of interviews, focus groups, surveys and artefacts
Statistical analysis of quantitative data (e.g. time to completion, survey data)
Human factors/ergonomics
Failure Modes Effect Analysis (FMEA) [15, 39] to risk stratify threats
Hierarchical task analysis [41] to understand task steps
Cognitive task analysis [41] to understand cognitive processes during tasks
Charting techniques [42, 44, 45], e.g. process charts, decision action guidelines
Mental workload assessment techniques [41], e.g. NASA Task Load Index
Situation awareness measurement techniques [41]
Team assessment methods (see also above)
Interface analysis [41], e.g. walkthrough analysis
Performance time assessment techniques [41], e.g. Critical Path Analysis
Design techniques [41], e.g. rapid prototyping, think aloud protocols
Quality improvement
Gathering information [45, 46], e.g. stakeholder analysis, benchmarking
Problem solving [45, 46], e.g. Five Whys
Understanding variation [42, 44, 45], e.g. statistical process control
Simulation-based Quality Improvement Tool (SQOIT) [15]
Incident reporting and root cause analysis [45, 46] (e.g. latent threats identified by ISS)
Cost-benefit analysis (60, 62)
  1. *Templates and instructions provided in the cited references, with additional selected examples used by the authors provided in the online supplemental appendix