Table 2 Instrument development process: activities and results

Round 1

Panel members (n = 10) screened video recordings from two simulation scenarios and identified events considered as signs of reduced or enhanced immersion. The identified events were sorted into a matrix and clustered in two steps.

The panel members (n = 10) identified 227 events that they consider as signs of reduced or enhanced immersion.

By a clustering process, the events were combined in 47 clusters. The 47 clusters were then combined to 11 triggers with 21 subheadings.

Round 2

The 11 triggers were sent back to the panel members (n = 10). The panel members were asked to classify the triggers as important or not important and adding potential new triggers.

10 triggers were scored as important by more than 80 % of the participants in the panel, and one trigger was deleted together with its two subheadings. No trigger was added to the list.

Round 3

Panel members (n = 10) were asked to classify triggers as signs of enhanced or reduced immersion. They also scored the importance of each trigger on a Likert scale: 1 = not important, 2 = not very important, 3 = possibly important, 4 = important and 5 = extremely important.

The panel members (n = 10) independently classified seven triggers as signs of reduced immersion and three as signs of enhanced immersion. All 10 triggers were classified as 4 or higher in importance by more than 80 % of the panel members. Three subheadings to the triggers were also added.

The panel members were also asked about the wording of the triggers.

Four triggers got new wordings.

Round 4

A new scoring of the trigger importance.

All 10 triggers were classified as 4 or higher in importance by more than 80 % of the panel members.

Content validity analysis

The panel members (n = 10) were asked to individually grade the relevance of the 10 triggers using a 4-point rating scale to rate the relevance of each individual trigger: 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant and 4 = highly relevant.

Trigger 6 reached an I-CVI = 0.8, triggers 2 and 10 I-CVI = 0.9 and the rest of the triggers I-CVI = 1.0.

The S-CVI which measure the content validity of the overall scale reached an acceptable standard of S-CVI/Ave = 0.96.

Inter-rater reliability analysis

6 raters were independently rating the same video recording from a simulation. The scenario was divided in intervals. In each interval, the immersion value was calculated. The IRR was assessed using an intra-class correlation calculation.

The ICC for the ISRI score was 0.92.