Table 4 Study ratings using Buckley's (modified) criteria
From: Observer roles that optimise learning in healthcare simulation education: a systematic review
Criteria (Yes, No, Unclear) | Bell | Harder | Hober | Kaplan | Lau | Smith | Stegmann | Stiefel | Thidemann |
|---|---|---|---|---|---|---|---|---|---|
Note: rater disagreement shown as two scores | |||||||||
Clear research question | U | Y | Y | Y | U | Y | Y | U | Y |
Subject group appropriate for study | Y | Y | Y | Y | Y | Y | Y | Y | Y |
Reliable and valid methods (qualitative or quantitative) used | Y | Y | Y | Y | Y/U | Y | Y | Y | Y |
Completeness of data (drop out, questionnaire response rate >60 %, attrition rate <50 %) | Y | Y | Y | Y | N | N | Y | N | Y |
Controlled for confounders or acknowledged if non RCT design | N | U/N | N | U/N | U | N | Y | U | Y |
Statistical and other analysis methods appropriate | Y | Y | Y | Y | Y | Y | Y | Y | Y |
Data justifies the conclusions drawn | Y | Y | Y | Y | N | Y | Y | U/N | Y |
Study could be replicated | Y/U | Y | Y | Y | U | N/U | Y | Y | Y |
Prospective study | Y | Y | Y | Y | Y | Y | Y | Y | Y |
Relevant ethical issues addressed | U | Y | Y | N | U | Y | Y | U | Y |
Triangulation of data | Y | Y | Y | Y | N | Y | Y | N | Y |
Total Score/11 (lowest score reported) | 7 | 10 | 10 | 9 | 3 | 8 | 11 | 5 | 11 |