- Open Access
Using stakeholder input to inform scenario content: an example from physiotherapy
Advances in Simulation volume 4, Article number: 20 (2019)
Simulated learning environments (SLEs) are being embraced as effective, though potentially costly tools, by health educators in a variety of contexts. The selection of scenarios, however, can be arbitrary and idiosyncratic.
We conducted a stakeholder audit to determine priorities for student learning which would inform scenario design. The process consisted of (1) the identification of stakeholders, (2) consultation with stakeholders to identify their priorities, (3) determination of priorities that could be addressed in the SLE being developed, and (4) incorporating these priorities into scenarios.
The identified stakeholders were the funding body, educational institution and discipline, regulatory agency, accreditation agency, external clinical placement providers, employers of new graduates, patients, and learners. Stakeholder input included a combination of surveys, consultation of online resources, and semi-structured interviews. Identified areas where student learning could be improved included (1) all students not having experience of all populations or ‘essential’ conditions, (2) situations where adverse events had occurred, (3) working with people from diverse backgrounds or those with psychosocial issues including those in chronic pain, (4) communication, (5) situation awareness, and (6) ethical issues.
Ten scenarios were developed considering the stakeholder input. Facilitator notes were written to ensure all facilitators addressed the areas that had been identified. Where possible, simulated patients, with diverse backgrounds, were hired to portray roles even though such areas of diversity were not explicitly written into the scenarios. Whilst the example concerns physiotherapy students within Australia, the principles may be applicable across a range of health disciplines.
Simulated learning environments (SLEs), which have been defined as techniques ‘to replace or amplify real experiences with guided experiences that evoke or replicate substantial aspects of the real world in a fully interactive manner’ , are being embraced as an effective, though potentially costly tool, by health educators in a variety of contexts. It has been suggested that the development of learning activities commences from an assessment of needs and progresses through identification of the learners and learning objectives to selection of educational and evaluation methods . For SLEs in the health professions, the order of development has sometimes been reversed assuming the method (simulation) and then considering the learners, learning objectives, and finally the scenario .
Many simulation centres exist in response to specific needs such as accreditation requirements [4, 5] or specialised skills training. In allied health professions, SLEs are frequently used to prepare for [6, 7], replace , or supplement  clinical placement experiences. In addition, the evolution to a ‘patient-focused’ health service creates the ethical drive to increased competency prior to direct patient contact [4, 5]. There is strong evidence, according to the Health Education and Training Institute’s (HETI) recent review , that, provided certain conditions are met, simulation can improve core knowledge and skills and replace up to 25% of clinical placement hours.
Within SLEs, scenario design is considered paramount [10, 11] and is one aspect that has a direct connection to achieving aims. Literature discussing scenario design stresses that simulation activities are part of an educational journey and as such, at a minimum, should be developed to integrate with curricular goals and graduate outcomes . It is also recommended  to incorporate the National Safety and Quality Health Service  into scenario design. Whilst curricular outcomes and patient safety are often considered [5, 13], strategies to determine the educational needs relevant to other stakeholders are infrequently discussed in the literature . We know of one example that has been described where scenario content was informed by the opinions of several levels of staff and students with respect to learning needs and patient care . In general, the selection of scenario content seems to be subjective and often idiosyncratic.
Response to individual and organisational needs has long been part of determining educational priorities for continuous professional development for qualified practitioners  and is driven by the belief that practice changes will be the result. A survey assessing education and training requirements of simulation professionals  found a clear need for health educators to have increased support in scenario design. Simulation scenario development was identified as a priority activity within their role, yet 53% of educators thought they needed additional training in this task.
With the purpose of providing health educators with potential direction in scenario design, we describe an approach to conducting a stakeholder audit from a broad range of stakeholders to inform not only the general simulation content, but also the detail within individual scenarios. The concept is described in relation to a specific situation—a simulated telemedicine experience for physiotherapy students undergoing a musculoskeletal clinical placement; it is hoped that the concepts will be useful in other disciplines and areas of practice.
The authors were in receipt of a federal grant to provide SLEs for entry-level students including those from physiotherapy with a primary objective of increasing clinical placement capacity and a secondary objective of supporting rural and remote clinical placements. The project was conducted at the Griffith University in Australia between the second semester of first-year and the first semester of the second-year of a two-year Masters of Physiotherapy program with an integrated clinical placement model. Students undertake a total of five 5-week placements, three of which would occur during these two semesters. The curriculum was accredited, so all essential content for graduates to be registered as physiotherapists was already present within the curriculum. The location of the SLE module within the curriculum was predetermined such that the funded SLEs would replace some hours of clinical placement in a musculoskeletal outpatient setting. The rural and remote priority was partially addressed by the SLEs being conducted via teleconference. Students would remain on their placement sites and be connected to simulated patients (SPs) and facilitators who were on campus.
The question that is the main topic of this paper is how to determine the content of the scenarios. An alternative approach to a formal needs analysis, the stakeholder audit has long been used in private industry as well as public enterprises  and consists of (1) identification of stakeholders, (2) determination of the stakes of each holder, (3) determination of how well the needs of each stakeholder are being met, and (4) adjustment to better meet the needs .
Methods and results
For the purpose of this audit, the steps were rephrased slightly to be (1) the identification of stakeholders from a multi-level perspective, (2) consultation with stakeholders to identify their priorities with respect to student learning needs, (3) determination of common or priority needs, and (4) establishing practical means of incorporating these learning needs into scenarios.
Identification of stakeholders
The identified stakeholders, stakes, and methods of gathering input are shown in Table 1. As described above, several aspects of the SLEs were determined by constraints of the funding and the location of the module in the curriculum. In other words, needs of some stake holders including the funding body and the logistics of university curriculum had already been considered. It was not considered to be feasible to consult patients on this occasion. The student consultations were with previous cohorts which provided an indication of their impressions of comparable clinical placements without the inclusion of the SLE.
Consultation with stakeholders
Consultations with stakeholders were divided into three main categories—reviewing online resources, an online survey to clinical educators, and semi-structured interviews with students and employers.
The Australian Physiotherapy Council (APC) threshold statements  and the Griffith Graduate Attributes  are policy documents available online. Since the physiotherapy program was accredited, it was considered to be compliant with APC priorities. The Griffith University graduate attributes (knowledgeable and skilled, with critical judgement; effective communicators and collaborators; innovative, creative, and entrepreneurial; socially responsible and engaged in their communities; culturally capable when working with First Australians; and effective in culturally diverse and international environments) extend beyond regulatory requirements to include qualities that are desirable in individuals both personally and within the broader community and were also considered to be addressed within the program. Nonetheless, these priorities and attributes were considered as additional influences that could inform decisions on scenario content.
Complaints to the regulatory body (AHPRA) and information from malpractice insurers provided documented examples of when real or perceived failures had occurred in professional practice and conduct. The most common issues identified from AHPRA were related to professional conduct including inappropriate advertising and boundary violations involving personal relationships. Two practice issues were also identified that related to identification of risk factors. First was a recommendation from the coroner related to a case of undetected deep vein thrombosis and the second was related to the risk of injury during unsupervised exercise.
The material from insurance companies related to minimising the risk of malpractice claims had three main aspects. First was ensuring appropriate practice standards, second the need for comprehensive and contemporaneous patient records, and finally appropriate responses to adverse events.
A link to an online survey was sent to the 13 facilities that hosted orthopaedic inpatient placements to be distributed to all clinical educators within that facility as well as those involved in supervising new graduates. Questions included identifying gaps in student experiences and new graduate skills. There were seven text-response questions and one to indicate the frequency of their ability to provide each of the experiences that have been deemed to be essential (Table 2). The text-based questions included essential or desirable experiences that the students may not always be exposed to in routine clinical placement, complications that the students must be able to identify, safety concerns, and challenging psychosocial considerations. An opportunity for additional suggestions was provided. There were 22 responses.
Only four out of the twenty-two respondents indicated that all ‘must know’ conditions were regularly experienced at their site. The most common gaps were spinal surgery, multi-trauma, and experience with patients across the lifespan—that is, students either primarily experienced paediatric patients or had minimal experience with a younger population. For eight of the fifteen items that the Queensland Orthopaedic Network identified as essential experiences, 90% of respondents indicated that they were always able to provide adequate exposure. The four items where less than 80% of the respondents indicated were ‘always covered’ were trauma, including peripheral, spinal, and multiple (35%); across the age continuum (61%); elective surgery, including upper limb, lower limb, and spinal (74%); and experience with slings and splints (78%).
The most common skills that were identified as lacking were (1) skills in patient care, e.g. getting in and out of bed and mobilising (n = 13), (2) communication—documentation and handover skills particularly between disciplines (n = 8), (3) risk minimisation (n = 7), and (4) understanding protocols (n = 4). Respondents identified a wide range of challenging psychosocial situations, but none addressed the second part of the question where they could suggest management strategies.
Respondents indicated a wide range of complications that students would benefit from experience of which were mostly related to aspects of psychosocial situations or risk minimisation as mentioned above. The aspects that the respondents felt students or new graduated ‘needed a stronger understanding of’ included clinical reasoning, risk minimisation, working with psychosocial issues, and communication particularly within the multidisciplinary team. No new topics related to scenario content were found in the final open-ended question.
After each clinical placement, students have a debriefing session which included discussion of areas where they would have liked to have been better prepared and experiences which had not been available to all students. In this instance, the information from previous cohorts who did not have access to the SLE was considered. Semi-structured interviews were conducted by one of the authors (SE) with six employers of new graduates including those from private practices and community health services who do not supervise students on orthopaedic inpatient placements. The interviewer knew approximately half of the employers who were interviewed. Student responses on gaps in areas of practice largely mirrored those found in the clinical educator survey. Students reported that they could have been better prepared for record keeping and communication, particularly in relation to their clinical educators.
Employers of new graduates identified communication skills and professional behaviour as the areas that could be improved. Their impression was that attitudes towards learning, willingness to request assistance, and lifelong learning skills were more important than specific clinical skills. A specific area of communication described by employers was in relation to job applications and interviews.
Identified gaps which could be addressed in a SLE
The two authors collated the information gathered from the stakeholders and presented this to the other two physiotherapy educators who were involved in developing the scenarios. The team reached consensus on which gaps that had been identified could be reasonably addressed within the SLEs that were being developed within the constraints of our funding agreement.
Gaps that were found through the consultative process that could be addressed in an SLE included (1) skills identified as lacking, (2) all students who did not have experience of patients with all populations or those with ‘essential’ conditions, (3) specific situations where adverse events had occurred, (4) working with people from diverse backgrounds or those with psychosocial issues including those in chronic pain, (5) communication including written (case notes and interprofessional communication) and verbal (with patients, educators, and other professionals), (6) situation awareness including how unexpected factors might impact on patient care, and (7) ethical issues including professional behaviour and boundary violations. In addition, although not actual gaps, there were areas where, when possible, content could be selected that was relevant to the Griffith University graduate attributes .
Some areas that were found were not considered to be able to be addressed in the current setting include (1) administrative aspects of private practice, (2) job applications and interviews, and (3) conforming with advertising regulations. Each of these areas, however, has subsequently been addressed in other areas of the curriculum.
Incorporating findings into scenarios
A total of ten scenarios were developed (see Additional file 1). Facilitator notes were written to ensure all facilitators addressed the topics that had been identified. Where possible, SPs with diverse backgrounds were hired to portray roles even though such areas of diversity were not explicitly written into the scenarios.
Populations and essential experiences
To meet the desire for student exposure to patients across the lifespan, scenarios were written to include patients across a wide age range. One scenario included a young patient with cerebral palsy. In this instance, we ensured authenticity by having an SP who was, at the outset of the project, a first-year university student with cerebral palsy and was able to portray an early teenage patient. Priorities of cultural competency were responded to by including patients from diverse socio-economic, cultural, and gender identity backgrounds. The exact nature of this diversity was not necessarily written into the scenarios but was a priority for recruitment of SPs for each iteration of the SLE.
Two scenarios were developed based on reports from AHPRA of adverse events. One scenario duplicated the situation recounted in the coroner’s report of an undiagnosed DVT. Contrary to the real-life setting, the SLE was structured with enough cues that the students would be expected to recognise the risk and ‘save’ the patient’s life. Debriefing included reference to the real-life situation where the patient had died and discussion of how the risk factors could have been missed. Another SP returned for their second treatment having sustained an injury whilst they were performing their prescribed exercises through no direct fault in their care. Debriefing could then consider risk minimisation strategies as well as the student response to an adverse event.
Patients with multi-trauma, spinal surgery, or in some instances surgery to peripheral joints were identified by clinical educators and students as areas where all students on clinical placement did not necessarily have experience. Scenarios included patients with all of these conditions.
Ability to work with patients with psychosocial issues
Working with patients with psychosocial issues is another area which is ideally suited to an SLE due to the safety of the environment and the ability to control the situations. Aspects that were written into the scenarios included patients with (1) two very different responses to chronic pain, (2) depression, (3) substance abuse, and (4) social isolation. Accurate portrayal of psychosocial issues requires clear conceptualisation and descriptions in the written scenario, but also rehearsal and role moderation for each SP. Debriefing strategies were described in the facilitator notes to guide discussion.
Scenarios were specifically designed to demand a variety of communication skills. The choice of a telehealth platform in itself challenged communication skills in a way not demanded in routine practice. Debriefing strategies were selected to encourage reflection  which utilised multiple perspectives (peer, facilitator, patient, and self) and were structured to target communication skills. Peers were encouraged to observe for the use of open-ended questions and active listening strategies. Facilitators addressed communications skills during the debriefing process, and time allowing, students were given the opportunity to repeat experiences to solidify their learning. Time permitting, the SPs provided feedback on communication ‘from the patient’s perspective’. Students wrote patient records from their encounters and were then provided with ‘example documents’ for the students to compare and reflect on. In addition, students were given access to videos of their personal performances to reflect on the interactions independently.
A specific area of communication described by employers was in relation to job applications and interviews. Although it was not possible to incorporate job applications into this SLE, a later separate simulation was developed to address this skill . To encourage interprofessional learning, interactions with dietetics, exercise physiologists, psychology services, and general practitioners, in both a responsive and proactive way, were integrated into the scenarios. Representatives from each discipline were consulted to ensure authenticity of scenarios design. For written communications, ‘example documents’ were provided to students after they had made an attempt at the task.
Situation awareness has been described as consisting of three components: perception of elements in the current situation, comprehension of current status, and projection of future status . It therefore does not require multiple events to be occurring simultaneously as is sometimes thought. The consultations in the current simulations were one-on-one and conducted via teleconference so there was little opportunity for multiple events to be occurring simultaneously. Nonetheless, it was essential for the students to be aware of and understand the patients’ situation beyond what was happening in front of them as well as project what the possible outcomes might be. Scenarios were structured to include a range of aspects that would be important to consider including family interactions, eligibility for workers’ compensation, and psychosocial issues. Prompts for the discussion of future implications of the patients’ situation were therefore included in the facilitator’s notes.
SLEs are ideally suited to working with ethical issues due to being able to present controlled situations and due to the safety of the environment. One scenario included a patient who had been affected by a boundary violation from a previous practitioner which would have triggered the need for mandatory reporting to AHPRA by the student. The need for professional behaviour was incorporated into facilitator notes for all scenarios. The pre-briefing for each session included a discussion of simulation being a safe environment and ‘what happens in simulation, stays in simulation’. Fortunately, the one exception that was explicitly stated was professional behaviour. One pair of students exhibited explicitly racist behaviour which they would not accept responsibility for during the debriefing sessions. As a result, the concerns were escalated with the facilitator’s impressions and a video recording of the interaction supplied to the relevant head of school.
A case is explored where a stakeholder audit process was used to inform scenario content. Although a range of priorities was found, there were few conflicting demands from different stakeholders. For example, the students and facilitators identified similar gaps in student experiences; the focus on record keeping was identified by clinical educators, employers, and insurance companies; and the need for situation awareness was implicit in input from the regulatory body, clinical educators, and employers.
For the future, deeper engagement with stakeholder classification literature may refine the identification and prioritisation process. In our context, including new graduates’ perspectives of learning need, from their very recent past, may have given us alternative insights. In addition, our frame of reference was very directed by the physiotherapy profession, a traditional approach to student education. It might be enlightening to seek the opinion of other disciplines who work regularly with our students or new graduates (e.g. nursing staff, doctors, other allied health). Similarly, the patients and their family’s thoughts might challenge us further.
Other learnings included the need to communicate the educational priorities to, at the minimum, the stakeholders immediately affected by the intervention. For example, whilst a priority from our funding was improving service to rural and remote Australia, neither the clinical educators nor the students appreciated the role of the telemedicine platform. As such, subsequent iterations used face-to-face simulation with similar scenarios. That is not to suggest that there is no value in telehealth for simulation, rather that better integration between educational and health technical systems would be necessary to make it more effective in the setting used in the current example.
A limited stakeholder audit was used to inform the content of simulation scenarios and structure of SLEs. Although the example in this paper was specific to physiotherapy students within Australia, it is hoped that the concepts could be useful across a range of health disciplines including in other countries.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Australian Health Practitioner Regulation Agency
Australian Physiotherapy Council
Simulated learning environments
Gaba DM. The future vision of simulation in health care. BMJ Qual Saf. 2004;13(suppl 1):i2–i10.
Østergaard H, Østergaard D, Lippert A. Implementation of team training in medical education in Denmark. Postgrad Med J. 2008;84(996):507–11.
Wright A, Moss P, Watson K, Rue S, Jull G, Mandrusiak A, Reubenson A, Connaughton J, Redmond C, MacIntosh S. A profession-wide collaboration to embed role-play simulation into Australian entry-level physiotherapy clinical training. Physiotherapy. 2015;101:e1047.
HETI: Simulation based education: professional entry student education and training. 2014.
Leape L. Unmet needs teaching physicians to provide safe patient care; 2010.
Mori B, Carnahan H, Herold J. Use of simulation learning experiences in physical therapy entry-to-practice curricula: a systematic review. Physiother Can. 2015;67(2):194–202.
Dalwood N, Maloney S, Cox N, Morgan P. Preparing physiotherapy students for clinical placement: student perceptions of low-cost peer simulation. A mixed-methods study. Simul Healthc. 2018;13(3):181–7.
Watson K, Wright A, Morris N, McMeeken J, Rivett D, Blackstock F, Jones A, Haines T, O’Connor V, Watson G. Can simulation replace part of clinical time? Two parallel randomised controlled trials. Med Educ. 2012;46(7):657–67.
Tuttle N, Bialocerkowski A. Developing student skills to actively engage in feedback conversations: a pilot study. Internet J Allied Health Sci Pract. 2017;15(4):10.
Nestel D, Harlim J, Smith C, Krogh K, Bearman M. Simulated learning technologies in undergraduate curricula: an evidence check review for HETI; 2014.
Alinier G. Developing high-fidelity health care simulation scenarios: a guide for educators and professionals. Simul Gaming. 2011;42(1):9–26.
Care ACoSaQiH. National Safety and Quality Health Care Standards (September 2012). Sydney: ACSQHC; 2012.
Aggarwal R, Mytton OT, Derbrew M, Hananel D, Heydenburg M, Issenberg B, MacAulay C, Mancini ME, Morimoto T, Soper N. Training and simulation for patient safety. BMJ Qual Saf. 2010;19(Suppl 2):i34–43.
Aboumatar H, Thomas P. Preparing medical students for their patient safety role on the wards: development and evaluation of a two day safety and teamwork course; 2009.
Grant J. Learning needs assessment: assessing the need. BMJ. 2002;324(7330):156–9.
HETI: Education and training requirements for simulation professionals in NSW Priorities identified from a survey of simulation professionals 2014 https://www.heti.nsw.gov.au/Global/NSS%20Simulation/EducationTrainingRequirementsSimulationProfessionalsNSW_HETI.pdf. Accessed 18 Jan 2017.
Roberts NC, King PJ. The stakeholder audit goes public; 1989.
Griffith University Graduate Attributes [https://www.griffith.edu.au/the-griffith-graduate].
Australian Health Practitioner Regulatory Agency. https://www.ahpra.gov.au Accessed 18 Jan 2017.
Physiotherapy practice thresholds https://physiocouncil.com.au/wp-content/uploads/2017/10/Physiotherapy-Board-Physiotherapy-practice-thresholds-in-Australia-and-Aotearoa-New-Zealand.pdf. Accessed 22 June 2019.
How to Avoid a Claim Against You https://riskhq.guildinsurance.com.au/articledetails/how-to-avoid-a-claim-against-you Accessed 18 Jan 2017.
HPSO. Physical therapy liability 2001–2010. In: Healthcare providers service organization; 2011.
Eppich W, Cheng A. Promoting Excellence and Reflective Learning in Simulation (PEARLS): development and rationale for a blended approach to health care simulation debriefing. Simul Healthc. 2015;10(2):106–15.
Kirwan G, Tuttle N, Weeks B, Laakso EL. Post-practicum strategies to translate clinical experience to attributes of employability – responding to graduate selection criteria. In: Billett S, editor. Augmenting student learning through post-practicum education; 2018.
Endsley MR. TToward a theory of situational awareness in dynamic systems. In: Salas E, editor. Situational Awareness. London: Routledge; 2017.
About this supplement
This article has been published as part of Advances in Simulation, Volume 4 Supplement 1, 2019. The full contents of the supplement are available at https://advancesinsimulation.biomedcentral.com/articles/supplements/volume-4-supplement-1.
The work presented in this manuscript was supported by the Department of Health, Australian Government, Award number 2013/002/051. The School of Allied Health Sciences Griffith University contributed to publication costs.
Ethics approval and consent to participate
Ethics approval was received from Griffith University HREC Ref # GU Ref No: PES/40/12/HREC.
Consent for publication
Consent for publication not applicable.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Edwards, S., Tuttle, N. Using stakeholder input to inform scenario content: an example from physiotherapy. Adv Simul 4, 20 (2019). https://doi.org/10.1186/s41077-019-0102-0
- Simulated learning environments
- Scenario selection
- Patient-centred simulation