Table 4 Post-experience survey results
From: The Debriefing Assessment in Real Time (DART) tool for simulation-based medical education
Overall rating of your experience using the DART tool (scales 1–7) | Overall rating of ease of using the DART tool (scales 1–7) | Overall opinion of usefulness of this tool for RATING the QUALITY of the observed debriefing (scales 1–7) | Overall opinion of usefulness of this tool as an adjunct to debriefer feedback (scales 1–7) | |
|---|---|---|---|---|
Likert ratings mean score (SD) | 5.5 (0.5) | 5.5 (0.5) | 5.0 (0.45) | 6.1 (0.54) |
Occurrences n (%) | Examples of suggestions for improvement/ relevant reflections | |||
Training process | 7 (28%) | • Calibration exercise prior to use of tool was helpful • A brief user guide with written examples on what constitutes a new statement and whether to consider a question/statement if repeated by same faculty would be helpful • Clarification on how to score multiple thoughts from the same speaker • Ambiguity with regard to how to score on the numbered rows in the tool • Hard to know where to split statements when scoring/tallying | ||
DART tool use | 10 (40%) | • Have a row of tallies for each topic within the debrief • Should we rate quality of questions? • Found the tool visually easy to use with the quick tick box system • Mental requirements of paying attention took away from ability to reflect on quality of debrief, tool requires attention to detail, and it limits the number done on any 1 day • Added section for free text would be beneficial for faculty feedback | ||
Applicability to practice | 8 (32%) | • Need to indicate what the DART scores mean • Unsure what the appropriate ratios are for effective debriefing • Not sure if scores correlate to good debriefing but could certainly help start a conversation between faculty and the tool focuses attention • Useful tool to provide objective feedback • Mixed evaluation of the debrief will be more beneficial for the debriefer; the DART tool can be recommended in conjunction with maybe the short version of the DASH tool • In a very junior group, more statements are required than a senior group, so interpretation of the result is required | ||