Table 1 (abstract A4). Classification of best practices based on the stage of assessment implementation

Planning phase

Right modality of simulation for the tasks to be evaluated

Real practice design to reflect the highest possible fidelity

Appropriate structure and resources standards of the assessment centre

Proper security measures to prevent breach of the exam content

Clear and user friendly scenarios for SPs, examiners and technical support staff

Determine the metrics of the assessment

Carefully chosen/designed assessment tools (checklists, global ratings) to facilitate high standard metrics

Raters should be trained and qualified based on a preset protocol

Measures to assure high reliability and validity e.g. increasing the number of scenarios, designing task-specific stations and standardizing the exams administration

Targeted metrics, e.g. Kappa >0.75, reliability, validity, accessibility, feasibility

During the assessment phase

Timing each station 5-10 minutes, in acute care medicine can be shortened to 5 minutes

Video and audio recording, as a quality measure, to be considered

Synchronizing and standardizing timing of the exam conducted in multiple centres

Availability of real-time technical support in all centres

Reinforcing measures to prevent cheating

Room designation during the examination, i.e. examiner, SP and/or technician

Post-assessment phase

Evaluation of the assessment process

Psychometric analysis of the scores to support the validity of the assessment

Explore for confounding factors that influence the outcome

Review the issues that appeared during the exam and actions accordingly

Develop continuous improvement plan

Update manuals and/or protocols

Demonstrate results and appeal process to the learners

Conduct remediation process as planned before