Table 4 Overview and characteristics of included studies

Boet et al. (2016)

[Toronto, Canada] [16]

To provide a narrative analysis of the content of debriefing discussion and determine the topics that facilitate reflection amongst interprofessional learners in SLDs and FLDs following a SLE.

Exploratory case-study approach.

Interprofessional teams comprising of 1 anaesthesia resident (n=36), 1 surgical trainee (n=36) and 1 theatre nurse (n=36) (total: n=108)

SLD: 17 teams

FLD: 19 teams

Following 10-min CRM team scenario, learners commenced either 20-min SLD or FLD. Both groups then undertook a second 10-min CRM scenario, followed by FLDs. Video playback available in all debriefings. SLDs used a form based on the OGRS, with learners being asked to ‘reflect on their CRM performance and on how it could be improved’.

All SLDs and FLDs were audio-recorded and transcribed. Data were analysed qualitatively using a constant comparison method and interpreted using a social constructivist framework. Authors used a consensus-based, iterative, inductive process to identify emergent themes.

(1) 3 emergent themes revealed topics that allowed learners to enter reflection were (1) the debriefing itself, (2) experience of the simulation model (including fidelity), and (3) performance, including assessment of CRM performance.

(2) SLD learners relied heavily on the OGRS form to guide their debriefing.

(3) FLDs followed a more precise structure and had directed conversation, whilst SLDs had more talking over and were less structured than FLDs.

Andrews et al. (2019)

[California, USA] [43]

To explore students’ perceived value of reviewing video recordings of, and on receiving faculty or peer feedback on, their NTS in an SLE.

2-arm prospective RCT.

3rd and 4th-year dental students (n=126)

Sample 1 (n=66):

SLD: n=45

FLD: n=21

Sample 2 (n=60):

SLD: n=43

FLD: n=17

Following 20-min individual scenario with SP, learners received 5-min feedback from SP, followed by 15-min group FLD. Learners then brought back between 11 and 78 weeks later, and randomly allocated to undertake either 1-on-1 FLD (up to 40 min) or SLD with 1 other learner (up to 90 min). Both groups watched video recordings of the scenario. Sample 2 SLD learners were trained to provide constructive feedback via a 60-min session (all other groups untrained). 9 reflection questions were given to learners in both groups to guide debrief.

(1) Posttest learners questionnaires^a (7 items via 4-point Likert scale) (2 additional items included for SLD group).

(2) Observer ratings of learner performance [ATOSCE] (sample 2 only).

(1) Learners rated FLDs significantly higher (p<0.05) than SLDs on most dimensions. However, both SLDs and FLDs were rated highly, with learners finding value regardless of the method.

(2) Perceived value did not differ by age, gender, class year or OSCE performance.

(3) Providing training for peer-feedback did not result in more favourable ratings for SLDs vs. FLDs

Ha (2020)

[Chungcheongbuk-do, South Korea] [44]

To determine the effects of written versus observed SLDs when using simulation with CBL and compare levels of satisfaction between the two groups.

2-arm non-equivalent control group pretest-posttest design.

3rd-year nursing students (n=69)

Written SLD: n=33

Observed SLD: n=36

Following 10-min team scenario, learners commenced either 20-min written SLD (using DD structure) or observed SLD (10 min watching another group undertake a scenario, prior to 10-min SLD to compare their scenario to that of the other group). Post SLDs, learners repeated the scenario, before completing questionnaires prior to undertaking a FLD.

(1) Observer ratings of learner performance pre- and posttest checklist^a (15 items across 6 domains).

(2) Posttest learner satisfaction with SBL questionnaire^a (20 items across 8 domains via 5-point Likert scale).

(3) Posttest learner Satisfaction with debriefing questionnaire^a (10 items via 5-point Likert scale).

(1) Clinical performance competency scores in both groups were significantly higher posttest vs. pretest.

(2) Communication was significantly higher in the observed SLD group vs. written SLD group (p=0.047). There were no significant differences in any other between the 2 groups.

(3) No significant differences in satisfaction with SBL (p=0.485) or debriefing (p=0.309) between the 2 groups.

Ha & Lim (2018)

[Chungcheongbuk-do, South Korea] [45]

To evaluate nursing students’ knowledge and confidence in preoperative nursing skills and satisfaction with debriefing and multimode simulation using SLDs compared with FLDs.

*Reported as a 2-arm quasi-experimental. However, authors report using random allocation for learners. It has therefore been included in this IR as a quantitative RCT.

3rd-year nursing students (n=124)

SLD: n=62

FLD: n=62

Following 20-min team scenario with a manikin and SP, learners commenced either 20-min written SLD (using checklist to structure debrief) or oral FLD (not described in detail). Post debriefings, learners completed questionnaires and a repeat knowledge test 2 weeks later (at which point SLD groups were offered an oral FLD).

(1) Pre- and posttest learner written knowledge assessment^a.

(2) Posttest learner self-confidence questionnaire^a (4 domains via 5-point Likert scale).

(3) Posttest learner satisfaction with multimode simulation questionnaire (22 items via 5-point Likert scale) (Ryoo et al., 2013).

(4) Posttest learner satisfaction with debriefing [DES].

(1) No significant difference in knowledge scores pre- to posttest in both groups (p=0.940).

(2) The self-confidence of preoperative nursing skills was statistically higher in the oral FLD group vs. written SLD group (18.81 vs. 17.85, p=0.010).

(3) No significant difference in learner satisfaction scores with multimode simulation (p=0.200) and debriefing (p=0.423) between the 2 groups.

Kim & De Gange (2018)

[Seoul, South Korea] [46]

To explore the effectiveness of SLDs and FLDs on nursing students’ knowledge, skills, self-confidence, and quality of debriefing following a preoperative care SLE using an SP.

2-arm non-equivalent control group pretest-posttest design.

3rd-year nursing students enrolled in operating room care course (n=57)

SLD: n=31

FLD: n=26

Following 20-min team scenario with SP, learners commenced either SLD (using a written GAS framework) or FLD. Post debriefings, learners repeated the scenario, before completing questionnaires. SLD learners could then ask instructors questions or for performance feedback.

(1) Pre- and posttest learner written knowledge assessment^a.

(2) Observer ratings of learner performance pre- and posttest checklist^a (18 items).

(3) Pre- and posttest learner self-confidence questionnaire^a (10 items via 10-point Likert scale).

(4) Posttest learner assessment of debriefing quality [DASH-SV].

(1) There were no significant differences in knowledge of preoperative care or self-confidence from pretest to posttest in either of the groups.

(2) Nursing skills for preoperative care (p< 0.001) was statistically higher in FLDs vs. SLDs.

(3) Quality of debriefing (p< 0.001) was statistically higher in FLDs vs. SLDs.

(3) There were no statistically significant differences in self-confidence (p=0.686) or knowledge (p=0.445) between the 2 groups.

Kündig et al. (2020)

[Bern & Basel, Switzerland] [47]

To test the effects of SLDs on resuscitation performance: hands-on time, coordination between chest compressions and ventilation and defibrillation.

2-arm RCT.

4th-year medical students (n=171)

SLD: n=81

No debriefing: n=87

Following 3-min CA team scenario, learners commenced either 3-min SLD (using written instructions on how to reflect) or 3-min no debriefing (this group performed X-ray interpretation tasks). After 3 minutes, learners commenced a second CA scenario.

(1) Observer assessment of learner performance of percentage hands-on time, coordination between chest compressions and ventilation, and defibrillation performance.

(1) Compared to the no debriefing group, learners in the SLD group showed higher performance gain in the second resuscitation scenario in percentage of hands-on time (6.21 percentage point increase, p< 0.001) and coordination between chest compressions and ventilation (15.0 percentage points, p< 0.001). However, the SLD group demonstrated a non-significantly reduced performance for defibrillation (minus 9 percentage points, p=0.312).

Oikawa et al. (2016)

[Hawai’i, USA] [48]

To determine if learner-assessed SPA and TPA scores were different when SBE occurred with SLDs or FLDs.

2-arm prospective, controlled, cohort intervention study.

Postgraduate year 1 doctors of varying specialities (n=57)

SLD: n=30

FLD: n=27

Following 5-min team scenario, learners commenced either 15-min SLDs (checklist-guided) or FLDs. Post debriefings, learners repeated the cycle 3 times. Following on from all 4 scenarios and debriefings, all learners attended an instructor-led course conclusion. The SLD group were instructed to independently complete a scenario-specific checklist prior to engaging in group discussion.

(1) Pre- and posttest learner assessment [TPA^b].

(2) Pre- and posttest learner assessment [SPA^b].

(1) Posttest scores for both TPA (p=0.13) and SPA (p=0.14) improved significantly compared to pretest scores in both groups.

(2) TPA scores were significantly higher for the SLD group (14.5) than the FLD group (13.3) (p=0.001) whereas there was no significant difference in SPA scores (p=0.05).

Rueda-Medina et al. (2020)

[Granada, Spain] [49]

To investigate the effectiveness of SLDs compared with FLDs or combined SLDs + FLDs, in terms of debriefing assessment, problem-solving process, and team effectiveness.

3-arm posttest-only experimental design.

Interprofessional teams of 2nd-year nursing (n=177), physiotherapy (n=39), and OT (n=36) students (total: n=252)

SLD: n=77

SLD + FLD: n=90

FLD: n=85

Following 15-min team scenario, learners commenced either 75-min SLD, combined SLD + FLD or FLD. In the SLD and combined SLD + FLD groups, learners independently completed a questionnaire (based on GAS framework), prior to commencing group discussions (either self-led or facilitator-led), using the questionnaire as a discussion guide. Whilst video playback of the scenario occurred in the SLD, it was unclear whether it was used in the combined SLD + FLD group. The GAS framework and video playback used in the FLDs. Post debriefings, learners completed questionnaires, with SLD group then attending a FLD.

(1) Posttest learner experience with debriefing [DES]. Facilitator aspects of the scale were modified to state ‘facilitated questions’ for SLD group.

(2) Posttest learner self-assessment of problem-solving abilities [PSI].

(3) Posttest learner assessment of team effectiveness [CATS].

(1) Nursing: the SLD + FLD group had a significantly higher CATS score (15.63) than either the SLD (13.91) or FLD group (13.71) (p< 0.001). There were no significant differences for DES or PSI between the groups.

(2) Physiotherapy: the SLD + FLD group had a significantly higher CATS score (13.50) than either the SLD only group (10.83) or FLD only group (10.36) (p< 0.009). There were no significant differences for DES or PSI between the groups.

(3) OT: there were no significant differences for DES, PSI or CATS between the groups.

Rueda-Medina et al. (2021)

[Granada, Spain] [50]

To compare the debriefing assessment and satisfaction perceived by nursing students who experience SLDs, FLDs or combined SLDs + FLDs.

3-arm randomised experimental design.

Nursing students (n=177)

SLD: n=58

SLD + FLD: n=68

FLD: n=51

Following a 15-min enteral-feeding team scenario, learners commenced either 75-min SLD, combined SLD + FLD or FLD. In the SLD and combined SLD + FLD groups, learners completed a questionnaire (based on GAS framework) prior to commencing group discussions (either self-led or facilitator-led), using the questionnaire as a discussion guide. GAS framework used in the FLDs. Video playback used in all groups. Post debriefings, learners completed questionnaires, with SLD group then attending a FLD.

(1) Posttest learner assessment of debriefing quality [DASH-SV].

(2) Posttest learner satisfaction with debriefing [CESS].

(3) Posttest learner satisfaction with debriefing [10-cm VAS].

(1) DASH-SV score in the SLD + FLD group was significantly higher compared with FLD group (p=0.03).

(2) DASH-SV score in the FLD was significantly higher compared with the SLD group (p=0.08).

(3) CESS score in the SLD + FLD group (146.60) was significantly higher than the SLD group (140.71) and the FLD group (136.07) (p=0.039).

(4) VAS score in the SLD + FLD group (9.25) was significantly higher than the SLD group (8.40) and the FLD group (8.37) (p=0.13).

Kang & Yu (2018)

[Jeju & South Gyeongsang, South Korea] [51]

To determine differences in the problem-solving process, team effectiveness, debriefing assessment, and debriefing satisfaction between SLDs + FLDs compared with FLDs only, and to determine if these are affected by number of SLD sessions.

2-arm non-equivalent control group pretest-posttest design.

4th-year nursing students (n=123)

SLD + FLD: n=60

FLD: n=63

Following 20-min team scenario, learners commenced either 30-min SLDs (using a questionnaire based on GAS framework and video playback) or FLDs. Post debriefings, learners repeated the cycle 4 times in sequence following on from other groups. The number of SLDs performed therefore varied depending on the sequence in which the groups undertook the scenarios (some groups undertook 5 SLDs, whilst others only had 1). Following completion of all 5 scenarios and debriefings, all learners attended a large group FLD.

(1) Pre- and posttest learner self-assessment of problem-solving abilities [PSPIA].

(2) Pre- and posttest learner assessment of team effectiveness tool (via 7-point Likert scale) (Lim & Kang, 2005).

(3) Posttest learner assessment of debriefing quality [DASH-SV].

(4) Posttest learner satisfaction with debriefing [10-cm VAS].

(1) The SLD + FLD group showed significant improvement in the problem-solving process (p<0.01) and debriefing satisfaction (p=0.02), but not in debriefing assessment (p=0.097) or team effectiveness (p=0.069) compared to the FLD group.

(2) Groups participating in a higher number of SLDs had significant improvements in problem-solving ability (p<0.001) and debriefing satisfaction (p<0.001). Furthermore, debriefing assessment scores and team effectiveness tended to increase with the number of SLD sessions, but non-significantly.

Lee et al. (2020) [52]

[Gangwon-do, South Korea]

To compare three debriefing methods (FLD, SLD, and video assisted SLD) by measuring academic self-efficacy, confidence in performance, self-assessed communication, and satisfaction.

Multi-site 3-arm quasi-experimental study using a pretest-posttest design.

Senior nursing students from 3 South Korean universities (n=146)

SLD: n=49

video assisted SLD: n=50

FLD: n=47

Following 20-min prematurity care team scenario, learners commenced either 90-min SLD (using a questionnaire structured via description, analysis, and application framework), video assisted SLD or FLD (using description, analysis, and application framework). In the video assisted SLD group, learners reviewed the full scenario recording (20 min), before providing written responses to the same questionnaire (15 min). This was followed by instructor feedback.

(1) Pre- and posttest learner academic self-efficacy assessment [ASES].

(2) Pre- and posttest learner self-confidence questionnaire (15 items via 5-point Likert scale) (Lee et al., 1991).

(3) Pre- and posttest learner self-assessed communication skills [GICC].

(4) Posttest learner satisfaction with debriefing questionnaire (16 items via 5-point Likert scale) (Otieno et al., 2007).

(1) Academic self-efficacy (p=0.001), confidence in performance (p<0.001), and self-assessed communication skills (p=0.007) all improved significantly pretest to postest in all 3 groups.

(2) There was no significant difference in posttest academic self-efficacy between the video assisted SLD group and the SLD and FLD groups (p=0.218).

(3) Posttest confidence in performance was significantly higher in the video assisted SLD group vs. the SLD (−0.07) and FLD (−0.33) groups (p=0.001).

(4) Posttest self-assessed communication skills were significantly higher in the video assisted SLD group vs. the SLD (−0.08) and FLD (−0.25) groups (p=0.007).

(5) Posttest satisfaction with debriefing methods were significantly higher in the video assisted SLD group compared to the SLD (−0.04) and FLD (−0.51) groups (p< 0.001). However, satisfaction was high in all groups.

Na & Roh (2021)

[Seoul, Gangwon & Chungcheong, South Korea] [53]

To compare the effects of SLDs and FLDs on cognitive load, achievement emotions, and the nursing performance of senior nursing students.

2-arm non-equivalent control group pretest-posttest design.

Senior nursing students (n=55)

SLD: n=26

FLD: n=29

Following 10-min hyponatraemia individual scenario, groups of 5 learners commenced either 50-min SLD (led by a volunteer student using GAS framework) or FLD (using GAS framework). Both groups contained a 20-min self-reflection with journal writing element (based on GAS framework) prior to group discussion. Following debriefing, the scenario was repeated.

(1) Pre- and posttest learner self-report on cognitive load [CLMT].

(2) Pre- and posttest learner self-report on achievement emotions [AEQ-Korean version].

(3) Observer ratings of learner performance [NPC].

(1) There were no statistically significant differences between the 2 groups regarding total cognitive load (p=0.437), nursing performance (p=0.559), or the achievement emotions.

(2) Nursing students in both groups showed a significantly higher overall cognitive load, higher positive and lower negative achievement emotions, and improved nursing performance after debriefing compared to before debriefing. However, when examining pre- and posttest score differences between the groups, there were no significant differences across all three measures.

Paige et al. (2021)

[Louisiana, USA] [54]

To investigate the efficacy of the Q-TAS, versus the TAS, as a formative assessment of teamwork to improve the quality of SLDs.

2-arm prospective comparative analysis.

Interprofessional teams of (typically) 2 senior medical (n=39), 2 nursing (n=8), and 2 nurse anaesthesia students (n=23) (total: n=70)

TAS SLD: n=35

Q-TAS SLD: n=35

Following theatre-based team scenario, learners commenced FLD. Post FLD, teams commenced a second scenario, followed by SLD. Sample 1 (2018) used the TAS tool and sample 2 (2019) used the Q-TAS tool to guide the SLDs (1 team member nominated as lead, who used the tool lead the discussion).

(1) Observer assessment of debriefing quality [OSAD].

(1) No significant difference in the overall quality of SLDs when using the Q-TAS (4.70) compared with the TAS (4.13) (p= 0.93).

(2) A statistically significant increase in quality of the analysis segment of SLDs when using the Q-TAS (4.92) compared with the TAS (3.83) (p= 0.23).

(3) Overall, regardless of tool used, quality of SLDs was rated highly (>4).

Schreiber et al. (2020)

[Pennsylvania, USA] [55]

To examine the use of SLDs for the purposes of assessing student perception of confidence with learning and performance related feedback during SBE.

2-part modified Pretest-posttest cross-over design.

Graduate OT students (n=37)

1st SLE: Active participants: (n=18)

Observers: (n=19)

2nd SLE (roles reversed): Active participants: (n=19)

Observers: (n=18)

Following 25-min team scenario with 2 active and 2 observing learners, learners commenced 60-min SLDs (led by the observing learners using a debriefing template). This process was repeated some months later but with the learner roles switched. A facilitator was present for the first 15 min to provide guidance.

(1) Pre- and posttest learner survey assessing self-confidence and competence with learning modalities^a (active participation compared with observing) and with giving and receiving feedback (6 items via 7-point Likert scale).

(1) Learners perceived benefit from both active participation and observation during SBE.

(2) In 1st SLE, observers believed more strongly that ‘I only learn by actively participating’ (1.05 vs. 1.17, p=0.025) but less strongly that ‘I am confident providing unbiased feedback to my peers’ (2.47 vs. 1.89, p=0.046) compared to active learners (difference not found in 2nd SLE once roles were switched).

(3) In 2nd SLE, observers believed more strongly that ‘feedback I give to my peers enhances my learning’ (1.89 vs. 2.10, p=0.037) compared to active learners (difference not found in 1^st SLE).

(4) Observers in 1st SLE disagreed more with the statement ‘I only learn by actively participating’ compared with observers in 2nd SLE (4.33 vs. 1.05, p=0.000).

(5) Active learners in 1st SLE, disagreed more with the statements ‘I learn by watching others’ (2.15 vs. 1.33, p=0.006), ‘I only learn by actively participating’ (3.60 vs. 1.17, p=0.000), and ‘feedback to peers enhances my learning’ (2.10 vs. 1.33, p=0.003) compared with active learners in 2nd SLE.

Tutticci et al. (2017)

[Queensland, Australia] [56]

To determine whether SLDs and/or SLDs + FLD differed from FLDs in assisting students to reveal critical reflection skills.

3-arm non-equivalent control group design.

Final-year nursing students (n=346)

SLD: n=110

SLD + FLD: n=158

FLD: n=78

Following a chest trauma team scenario with 4 active and 4 observing learners, learners commenced either 20-min SLD (led by randomly allocated learner), SLD + FLD (co-facilitated by randomly allocated learner) or FLD. All groups required their respective facilitators to use the 5 ‘Rs’ reflective framework and checklist- reporting, responding, relating, reasoning, and reconstructing (Bain et al., 2002). All facilitators were directed to online training resources to prepare for the role of facilitator (with option to opt-out).

(1) Posttest learner assessment of reflective thinking [RTI].

(2) Posttest learner Critical Reflection Self-Efficacy [single item 0–100 VAS].

(3) Posttest learner self-confidence in coping ability [GSES].

(4) Observer assessment of debriefing quality via checklist^a (9 items using yes/no, including 2 items with frequency score).

(1) FLD and SLD + FLD groups had a significantly higher RTI scores vs. SLD groups (p=0.006).

(2) There was no significant difference in the Critical Reflection Self-Efficacy VAS scores (p= 0.201) or the GSES scores (p= 0.933) between the 3 groups.

(3) Debriefing checklist adherence ranged from 10.9 to 92.7% across the 9 items. No data on group adherence to data is presented.

Curtis et al. (2016)

[Queensland, Australia] [57]

To evaluate peer-to-peer facilitated student led mid-level fidelity simulation experiences.

Single group posttest design.

2nd and 3rd-year nursing students (n=509)

Following 10-min team scenario in which learners played 4 specific roles (handheld device operator, nurse, physician, and observer), learners commenced 5-min SLD (led by learner playing role of handheld operator, using 3-question approach). Following completion of all 4 scenarios and debriefings, all learners attended a large group 30-min FLD.

(1) Posttest learner satisfaction questionnaire^a (16 items via 6-point Likert scale).

(2) Posttest learner self-confidence questionnaire (6 items adapted from SCLS).

(1) Learners self-reported high satisfaction in learning with the SLE (4.42).

(2) Learners reported high self-confidence in clinical skills after exposure to the SLE (4.14).

Boet et al. (2013)

[Toronto, Canada] [15]

To test the relative effectiveness of operating room SLDs compared with FLDs for learning CRM.

Two-arm RCT using repeated measures design.

Interprofessional teams of 1 anaesthesia resident (n=36), 1 surgical trainee (n=36) and 1 theatre nurse (n=36) (total: n=108)

SLD: 17 teams

FLD: 19 teams

Following 10-min CRM scenario, groups commenced either 20-min SLD or FLD. Both groups then undertook a second 10-min CRM scenario, followed by FLDs. Video playback available in all debriefings. SLDs used a form based on the OGRS, with learners being asked to ‘reflect on their CRM performance and on how it could be improved’.

(1) Pre- and posttest observer assessment of learners [TEAM].

(2) Transcripts of debriefings analysed using qualitatively using a constant comparison method. Detailed qualitative results published separately (Boet et al., 2016).

(1) Pre- to posttest team performance significantly improved regardless of debriefing method (p=0.008).

(2) There was no significant difference in the degree of improvement between the groups (p=0.52).

(3) Use of scenario video playback was similar in both groups.

(4) Similar themes were discussed in both groups.

Quick (2016)

[Minnesota, USA] [42]

To gain insight into the nature of the role of self- and peer-assessment in the development of dental students’ reflective practice skills with the use of an SP experience.

Single group posttest analysis design.

4th year-dental students

(n=32)

Pairs of learners undertook 4 scenarios with an SP (domestic violence, unrealistic expectations, informed consent, and breaking bad news). Learners rotated to be the active and observing learner twice each. The observer completed a worksheet during the encounter with the SP providing feedback to the active learner post-scenario. Following completion of all 4 scenarios, all learners attended a large group 35-min FLD. Learners were then re-paired such that each new pair had been the active learner with the same SP and scenario. They independently reviewed their own and their peers’ videos, before commencing a SLD (elements are not well defined beyond ‘meeting to discuss their feedback’).

(1) Posttest learner self- and peer-assessment forms, including quantitative^a (3 items via 1–10 scale) and qualitative (open-ended questions) data.

(1) 5 performance themes emerged: personal affect, verbal communication, professional demeanour, relationship-building, and patient management.

(2) 2 student learning themes emerged: application and knowledge, ways to change, and impressed with peer/increased confidence in self.

(3) Data showed that peer assessment ratings were consistently higher across the 3 items vs. self-assessments (case management: 8.09 vs. 7.25, relationship-building: 7.96 vs. 7.26, cue recognition: 8.45 vs. 7.90).