Table 1 All data extracted from studies examined in this paper. In addition, we have calculated theoretical false positives (FP) and theoretical false negatives (FN) and provided the absolute difference between observed and theoretical FP and FN.
Ref. | Number of novices | Novices’ score, mean (SD) | Number of experts | Experts’ score, mean (SD) | Pass/fail cut-off score | Novices passed (observed FP), n (%) | Calculated theoretical FP, % | Absolute difference in FP | Experts failed (observed FN), n (%) | Calculated theoretical FP, % | Absolute difference in FN |
|---|---|---|---|---|---|---|---|---|---|---|---|
[6] | Data only available for one group | ||||||||||
[10] | 20 | 244 (88) | 20 | 446 (52) | 358 | 2 (10.0%) | 9.8% | 0.2% | 2 (10.0%) | 4.5% | 5.5% |
[11] | 13 | 38.6 (27.3) | 13 | 0 (9.1) | 15.5 | 2 (15.4%) | 19.9% | 4.5% | 1 (7.7%) | 4.4% | 3.3% |
[12]a | 10 | 1.5 (0.4) | 10 | 4.4 (0.4) | 3 | 0 (0.0%) | 0.0% | 0.0% | 0 (0.0%) | 0.0% | 0.0% |
[12]b | 10 | 1.8 (0.2) | 10 | 3.9 (0.5) | 2.5 | 0 (0.0%) | 0.0% | 0.0% | 0 (0.0%) | 0.3% | 0.3% |
[13] | 14 | 0.27 (0.065) | 14 | 0.65 (0.117) | 0.42 | 0 (0.0%) | 1.1% | 1.1% | 0 (0.0%) | 2.5% | 2.5% |
[14] | No numbers on pass/fail | ||||||||||
[15] | No numbers on pass/fail | ||||||||||
[16] | Data only available as median and range | ||||||||||
[17]c | 11 | 93.1 (73.4) | 10 | 459.7 (147.5) | 235 | 1 (9.1%) | 2.7% | 6.4% | 0 (0.0%) | 6.4% | 6.4% |
[17]d | 11 | 41.4 (35.5) | 10 | 106.9 (102.5) | 93 | 1 (9.1%) | 7.2% | 1.9% | 7 (70%) | 44.6% | 25.4% |
[18] | 26 | 333 (96) | 11 | 497 (52) | 422 | 5 (19.2%) | 17.7% | 1.5% | 1 (9.1%) | 7.5% | 1.6% |
[19]e | 15 | 7.2 (1.1) | 10 | 27 (3.2) | 15.5 | 1 (6.7%) | 0.0% | 6.7% | 1 (10.0%) | 0.0% | 10.0% |
[19] f | 15 | 0.32 (0.31) | 10 | 2.48 (1.09) | 0.79 | 1 (6.7%) | 6.5% | 0.2% | 0 (0.0%) | 6.1% | 6.1% |
[20] | 10 | 30 (32) | 10 | 76 (10) | 58 | 0 (0.0%) | 19.1% | 19.1% | 1 (10.0%) | 3.6% | 6.4% |
[22]g | 8 | 0.098 (0.074) | 6 | 0.240 (0.037) | 0.19 | 1 (12.5%) | 10.7% | 1.8% | 0 (0.0%) | 8.8% | 8.8% |
[23] | Did not use contrasting groups | ||||||||||
[24] | 11 | 2.7127 (2.25645) | 10 | 0.7890 (0.39156) | 1.51 | 5 (45.5%) | 29.7% | 15.8% | 0 (0.0%) | 3.3% | 3.3% |
[25] | No numbers on pass/fail | ||||||||||
[26] | No numbers on pass/fail | ||||||||||