Table 3 Example of the development of an assessment tool for the observation of teamwork in simulation [48]
From: Simulation-based summative assessment in healthcare: an overview of key principles for practice
Source of validity | Method | Judgment criteria | Results |
|---|---|---|---|
Content | 1. Description of the Clinical Teamwork Scale (CRM scale) Development | Literature review Scale already used in another field (aeronautics) | 15 items 5 categories 1 overall skill score |
Response process | 1. Relevance of items 2. weighting items 3. Raters’ training (moderate) | 1. Precise description of each item 2. Quantitative criteria 3. Qualitative criteria 4. CRM principles | 1. Ratings aid table 2. 0 to 10 or 0/1 Descriptive levels: not relevant/unacceptable/poor/average/good/perfect |
Internal structure | 1. Built-in validity 2. Scale usability 3. Reproducibility | 1. Distribution of scores from the preset level 2. Number of items filled in full 3. interrater concordance, the correlation between overall score and categories (Kappa, Kendall, Pearsons, ICC) 4. Variance of each category | 1. Score tailored to each level 2. Easy-to-use scale (little loss of information) 3. correlation between raters 4. Variation in scores between scenarios sources of error |