Skip to main content

Advertisement

Table 1 All data extracted from studies examined in this paper. In addition, we have calculated theoretical false positives (FP) and theoretical false negatives (FN) and provided the absolute difference between observed and theoretical FP and FN.

From: Contrasting groups’ standard setting for consequences analysis in validity studies: reporting considerations

Ref. Number of novices Novices’ score, mean (SD) Number of experts Experts’ score, mean (SD) Pass/fail cut-off score Novices passed (observed FP), n (%) Calculated theoretical FP, % Absolute difference in FP Experts failed (observed FN), n (%) Calculated theoretical FP, % Absolute difference in FN
[6] Data only available for one group
[10] 20 244 (88) 20 446 (52) 358 2 (10.0%) 9.8% 0.2% 2 (10.0%) 4.5% 5.5%
[11] 13 38.6 (27.3) 13 0 (9.1) 15.5 2 (15.4%) 19.9% 4.5% 1 (7.7%) 4.4% 3.3%
[12]a 10 1.5 (0.4) 10 4.4 (0.4) 3 0 (0.0%) 0.0% 0.0% 0 (0.0%) 0.0% 0.0%
[12]b 10 1.8 (0.2) 10 3.9 (0.5) 2.5 0 (0.0%) 0.0% 0.0% 0 (0.0%) 0.3% 0.3%
[13] 14 0.27 (0.065) 14 0.65 (0.117) 0.42 0 (0.0%) 1.1% 1.1% 0 (0.0%) 2.5% 2.5%
[14] No numbers on pass/fail
[15] No numbers on pass/fail
[16] Data only available as median and range
[17]c 11 93.1 (73.4) 10 459.7 (147.5) 235 1 (9.1%) 2.7% 6.4% 0 (0.0%) 6.4% 6.4%
[17]d 11 41.4 (35.5) 10 106.9 (102.5) 93 1 (9.1%) 7.2% 1.9% 7 (70%) 44.6% 25.4%
[18] 26 333 (96) 11 497 (52) 422 5 (19.2%) 17.7% 1.5% 1 (9.1%) 7.5% 1.6%
[19]e 15 7.2 (1.1) 10 27 (3.2) 15.5 1 (6.7%) 0.0% 6.7% 1 (10.0%) 0.0% 10.0%
[19] f 15 0.32 (0.31) 10 2.48 (1.09) 0.79 1 (6.7%) 6.5% 0.2% 0 (0.0%) 6.1% 6.1%
[20] 10 30 (32) 10 76 (10) 58 0 (0.0%) 19.1% 19.1% 1 (10.0%) 3.6% 6.4%
[22]g 8 0.098 (0.074) 6 0.240 (0.037) 0.19 1 (12.5%) 10.7% 1.8% 0 (0.0%) 8.8% 8.8%
[23] Did not use contrasting groups
[24] 11 2.7127 (2.25645) 10 0.7890 (0.39156) 1.51 5 (45.5%) 29.7% 15.8% 0 (0.0%) 3.3% 3.3%
[25] No numbers on pass/fail
[26] No numbers on pass/fail
  1. Abbreviations: SD standard deviation, FP false positives, FN false negatives
  2. An absolute difference of > 5% between the observed and theoretical FP and FN are marked in italics
  3. aTransabdominal novices
  4. bTransvaginal novices
  5. cData from case 1
  6. dData from case 2
  7. eData from the virtual reality model
  8. fData from the physical model
  9. gMean and standard deviation are estimated from median and interquartile range