Non-Technical Skills Bingo—a game to facilitate the learning of complex concepts

Non-technical skills (NTS) are an integrated part of the abilities that healthcare professionals require to optimally diagnose and treat patients under the complex conditions of healthcare. NTS in healthcare have been defined as “the cognitive and social skills that complement technical skills and medical knowledge in task performance” [1]. The use of NTS aims to improve patient outcome through both an increase in patient safety and operational efficiency [2–4]. NTS are applied widely in conceptual and empirical papers, including those pertaining to simulation in healthcare [5–10]. There are several frameworks to describe NTS, for example, crisis resource management (CRM) [10] or the TeamSTEPPS framework [11]. The term “non-technical skills” is under debate [12]; however, due to its current widespread use, we have opted to use it here for the purposes of this paper.

We refer to the Danish variant of the Anaesthetists’ Non-Technical Skills system, ANTSdk [6], in its English translation (Fig. 1). ANTSdk comprises the four categories “situation awareness”, “decision making”, “leadership”, and “team work”. Each category has a number of underlying elements. Each element is in turn described by numerous behavioural markers, examples of positive and negative behaviours. ANTSdk will be used to illustrate the game-based exercise, called NTS Bingo. It is possible to adapt the NTS Bingo to other frameworks, disciplines, and professions. We use the discussion of ANTSdk and the application of NTS Bingo to anaesthesia as an example.

Several societies for anaesthesiology, intensive care medicine or medical education have included NTS training as crucial for patient safety [13–15]. The European Resuscitation Council [16] also includes NTS into their recent resuscitation guidelines. NTS are increasingly becoming part of medical licencing [8]. In Denmark, NTS sub-dimensions are an integrated part of the so-called competence cards that anaesthesiologists are required to collect as part of the portfolio on which their licencing is based. The Danish and Dutch systems, for example, build on the CanMeds Roles [17]. These roles are “medical expert”, “communicator”, “leader”, “health advocate”, “collaborator”, “scholar”, and “professional”. NTS can naturally be integrated into these roles.

One of the main functions of NTS frameworks is to stimulate discussion around how healthcare professionals can behave more safely and effectively. All healthcare professionals use NTS, even if they are unaware of them. The NTS frameworks are intended to help their users reflect on their existing NTS with the intention of being able to retain what works well and improve those actions that are not effective. Such reflection requires that learners understand how to apply the different ANTSdk categories and elements to their behaviours in different clinical contexts.

Learning to use, teach, and assess NTS are all complex endeavours. A new language underpinned by new concepts is being introduced. Concepts are more than terms [18]. Concepts structure what we perceive and how we orient ourselves in the world. Their importance is also emphasized in many debriefing models used in healthcare simulation, for example, the mental frames that should be investigated during debriefings [19]. Often we are not directly aware of the concepts we hold—we take many things for granted (the laws of gravity for example). NTS Bingo tries to stimulate reflection on the concepts that learners hold by asking participants to apply the concepts in a context that participants are not familiar with. The terms used in ANTSdk can seem deceptively simple and may create the illusion of a deep conceptual understanding, whereas in fact users might only have processed the terms on a superficial level, without forming robust concepts underlying the terms [20, 21]. The terms used to describe ANTSdk categories and ANTSdk elements often overlap in the complexity of the clinical world, which can present challenges when trying to get a grasp of the underlying concepts. NTS Bingo was created with the aim of supporting the formation of NTS concepts and improving the understanding of NTS terms.

We suggest that NTS Bingo can support simulation practice by helping simulation facilitators to improve their own understanding of NTS. This would be beneficial for a scenario design, as facilitators might then be able to design NTS challenges more systematically into scenarios [22, 23]. For debriefing, such an improved understanding of the NTS concepts will help simulation facilitators to guide the discussions towards NTS concepts [24–26]. An improved understanding of NTS will help simulation course participants to perform better during the simulation and also promote the development of better care in the clinical setting. A shared understanding of NTS between facilitators and participants should facilitate the discussions and mutual understanding which in turn should lead to more effective and deeper reflection during debriefing [27, 28].

One key feature of ANTSdk relevant in this context is that the elements in the ANTSdk categories “leadership” and “teamworking” tend to be more accessible to direct observation, whereas the ANTSdk elements in “situation awareness” and “decision making” tend to require more inference about mental processes based on observed behaviour. NTS Bingo could contribute to making this distinction clearer to simulation facilitators.

We did not formally evaluate NTS Bingo but conducted approximately 30 sessions during conferences and faculty development programmes as proof of concept. Oral feedback for those sessions was positive. Several groups working with NTS in simulation have adopted NTS into their practice.

In this section, we provide theoretical considerations to explain why we consider NTS Bingo an effective strategy to learn about NTS. We focus our considerations on “concept learning”, a study area within cognitive psychology [29]. We first discuss how “concepts” are discussed in the literature. We then describe how the acquisition of concepts can be supported.

Concepts provide the knowledge upon which a person is able to do a categorization, which is the process of recognizing what things, actions, or events belong together [29]. Concept learning is more than merely knowing a certain expression (e.g. recalling the steps in an algorithm). Grasping the concept, for example, that the pitch in the saturation tone in a patient monitor changes with oxygen levels in the blood of the monitored patient allows a person to distinguish patients with good oxygen saturation from patients with oxygenation problems. For the sake of simplicity, we will use the word “entity” to summarize things, actions, or events. Thus, concepts are used to mentally categorize entities as examples or non-examples of a mental category. The ANTSdk categories and elements are examples of such mental categories. They can be used to categorize human action in the context of anaesthesia in relation to NTS. In order to clearly distinguish the two senses of the word “category” in this paper, we will use ANTSdk categories to refer to the words in the ANTSdk system. We will use mental category to describe the cognitive structure of facilitators and participants.

The more familiar a person is with a given concept, the better this person will be able to correctly classify entities as an example or non-example of a mental category—in a variety of contexts and where the entities do not have clear features to serve as a basis for classification (e.g. an atypical case of a disease, unfamiliar communication techniques) [30].

Concepts are not solely verbal [31]. The mere word is not enough to describe the concept (e.g. “teamwork”, “situation awareness”, “leadership”, “love”, or “reality”). Grasping a concept involves also “feelings, beliefs, and other affective components” [31, 32]. This is a part of the explanation why several people might use a certain expression fluidly in a conversation in shared understanding and yet, when asked to describe their concept underlying the expression, they might describe it very differently.

A strategy to support concept learning has been described [33]. In a first step, relevant features of a concept can be discussed, for example, by stating a definition. In a second step, learners can be presented with entities that are clear examples of a mental category (e.g. actions that are clear examples for one of the ANTSdk elements). These entities should be very clearly examples of the ANTSdk element and should not contain many irrelevant or accidental features. In a next step, learners can be presented with non-examples for a mental category (e.g. actions that clearly do not belong into the ANTSdk element under discussion). Then learners can gradually be introduced to more ambiguous entities that are not clearly an example or non-example of the ANTSdk element under discussion. Finally, concepts can be applied in new contexts to support their generalization (e.g. using the ANTSdk element to categorize non-medical entities as examples or non-examples for the element). During generalization, the learner becomes clearer about the difference between the essential features of a concept and any arbitrary features. Generalization of a concept is needed to also categorize atypical entities correctly. Consider the ANTSdk element “identifying options” as an example. In many cases, it would be irrelevant whether decision options are communicated orally or in writing, as long as the different options are considered.

Typical challenges in concept learning are misconceptions, overgeneralizations, and undergeneralizations: Misconceptions describe instances where learners form an understanding of a concept that is not in line with accepted definitions [31]. In undergeneralization, learners would categorize an actual example entity wrongly as non-example for a mental category. In overgeneralization, learners would categorize an actual non-example entity wrongly as example for a mental category [33].

Consider the ANTSdk element “gathering information”. In a teaching session, this can be defined as “vigilantly and systematically seeking out information about the situation” [6]. Then learners could discuss entities from their own practice area that they see as clear examples of “gathering information”. Participants could then discuss what entities that they see as non-examples for “gathering information”. Then participants could be asked to describe examples and non-examples for “gathering information” from a variety of contexts, for example, from their own professional past or private life. Finally, the group could discuss which entities might be difficult to classify as either examples or non-examples for “gathering information” to explore the borders of the concept. Learners could discuss for “gathering information” that considering all sources of information is not helpful because of a danger of information overload. Ignoring important information sources, on the other hand, might provide an incomplete picture of the situation and that may result in wrong treatment decisions.

How to concretely implement such a concept teaching sequence depends on the learners and their previous knowledge. The video examples chosen for NTS Bingo can be based on their anticipated clarity as examples or non-examples for ANTSdk categories and ANTSdk elements. Some of the clips we suggest below contain clear examples for some ANTSdk elements but not others. They thus provide good training opportunities to categorize entities as example or non-example of ANTSdk elements.

NTS Bingo loosely builds on the original Bingo idea, where participants get cards with numbers printed onto them. There are a variety of different cards and not all possible numbers are contained on each card. A game leader uses some kind of random mechanism to draw numbers from a pool of possible numbers. Players mark the drawn number, if it is on their own card(s). If a player can mark a full vertical, horizontal or vertical line on his or her card, the player calls “Bingo” and wins a prize, when she or he is the first to do so in the round.

In NTS Bingo, the numbers on a card are replaced by ANTSdk elements. The drawing of numbers and their announcement is replaced by showing video clips to participants, who “mark” an element by writing down observations from the video they are shown. Thus, there is a key difference to real Bingo: Players of NTS Bingo need to identify their “hits” by themselves (subjectively)—using the mental concepts they formed based on the ANTSdk elements to categorizing entities, typically actions, into ANTSdk elements. The aim of NTS Bingo is for the players to fill in their NTS Bingo card as quickly as they can with behavioural examples of people they see in a video clip. Entities that can be categorized typically address not only the interaction between people but also how people interact with devices, machines, or other elements of the environment.

NTS Bingo is suitable for groups of four to eight players and a game leader. It is possible to have several parallel groups in settings with a larger number of participants, but still all watching the same video clip. One member of each group is the game leader for each round and this task can rotate. It is desirable, however, that the game leader is experienced in NTS. The game leader should be able to point out potential misconceptions and over- and undergeneralizations the players might hold, and thus would need some NTS expertise.

Consider a fictitious example. You see a woman driving a car on a freeway. She is approaching an exit. She almost passed it, and then she looks surprised. She checks the mirror shortly, brakes hard, turns with screeching tires and just barely makes the exit.

Recognizing the exit could be noted under “gathering information”. The same would go for checking the mirror.

The surprised look on her face might be an indicator for a lack of “planning ahead” or a lack of “prioritizing”, making inferences from observable behaviour to inner states.

The first player to have an example for all ANTSdk elements on his or her card calls “Bingo”. This often happens only after the video has ended. The game leader keeps track of the sequence in which the players call “Bingo”, if more than one call of “Bingo” is made. It can be beneficial to give the group a bit of time after the video is finished until all either have a Bingo or until they indicate that they have given up or that they have not found any more examples in the video clip.

The game leader then asks the person who first called the “Bingo” to “defend” their examples written on their card. The aim of this “defence” is to stimulate a discussion of how the video relates to the elements on the card. It is good to involve all players in the discussion. The defence is meant to stimulate discussion, is highly subjective, and ideally handled in a positive, yet constructively challenging, atmosphere. The game leader and other players should request convincing explanations as to why the caller thinks that her or his observation could be seen as an example for an ANTSdk element. The game leader needs to balance an early acceptance of a defence in order to keep the flow in the game and in order to preserve a positive atmosphere on the one hand with requesting a more elaborate discussion of each example on the other hand. Early acceptance of the defence might be related to positive emotions for the defender and yet might prevent deeper analysis. On a theoretical basis, the correct classification of an entity as non-example of an ANTSdk element is an important step in concept learning. The more arguments are exchanged to classify an entity as an example or non-example of an ANTSdk element, the better should the concept learning be. On the other hand, participants might also lose interest, when too many details are requested. We count on the skills of the game leader to adjust the guidance of the discussion here.

Above we discussed in general why NTS Bingo might help in concept learning. Here we discuss the content side of the video clips. It might be beneficial to remove the healthcare contents in some learning sessions on NTS, because by removing the healthcare content, it appears to be easier to concentrate on the NTS aspects. Out-of-context discussions can reduce potential overload from a mix of clinical considerations and trying to grasp new concepts at the same time [36]. The discussion of NTS in an out-of-context situation may increase psychological safety because misconceptions can be clarified in a situation, where the professional self-image is not at stake. Participants from various backgrounds can participate on “equal grounds”. The video clips that we suggest using in NTS Bingo often use caricature or drama elements and thus make certain points more clearly than documentary clips. Alienation, seeing an interaction in different contexts, might facilitate concept learning and applicability in real healthcare situations, because the relevant features of a concept become clearer by using it in such diverse categorization tasks [37]. On the other side, such exercises require to help participants make a “far transfer” [34] from the learning situation to their clinical work: participants will have to adapt what they learned during the game to their clinical work.

In many of the videos that work well with NTS Bingo, one or more persons are depicted in situations that most of us would not necessarily share with others. In most of the videos we use, this is staged and nobody gets hurt or embarrassed for real. However, some videos also depict real-world challenges and we strive to find a balance between having a video that is usable to make a strong point and an ethical treatment of videos of others. The legal and cultural perspective in the selection of the video clips needs to be taken into account, depending on the country in which NTS Bingo is played. What might count as a funny example in one context might violate cultural norms in a different context.

The definition of the categories and elements in ANTSdk and similar systems are not mutually exclusive, and the elements are related to each other. Consequently, many video clips will be relevant for several elements and one element can be illustrated with different video clips. Theoretically speaking, this suggests that NTS Bingo is better suited for learners with an already existing basic understanding of NTS, who are able to do basic categorizations of entities as examples or non-examples of ANTSdk elements. Without such a basic understanding, it might be difficult to distinguish examples from non-examples of an ANTSdk element.

Ambiguous video clips are very interesting—those that could only be interpreted after knowing the motives of the acting persons. This is often the case in clips with little context information and where the persons involved do not talk about their motives but mainly act. Such clips can stimulate elaborate reflections regarding their interpretation.

As mentioned earlier, we did not evaluate NTS Bingo formally. Here we describe some take home messages that we drew from our experience of running NTS Bingo sessions:

For some participants, it takes one or two rounds to get familiar with the procedures of NTS Bingo. We therefore systematically include a trial example, where we demonstrate the procedures in the session introduction in a step-by-step fashion.

Some participants do not immediately see the point of using non-medical video clips. On the other hand, we also received feedback that this particular aspect was valuable for participants—especially in combination with very different clips that allow participants to apply the NTS concepts in different contexts.

A potential challenge is the need to balance the conceptual precision around the NTS concepts while keeping a flow in the game during the defence. As described above, we would accept some degree of misconception, under- and overgeneralizations of the NTS concepts in order to stimulate participants’ reflections and in order to keep the flow in the discussion. We recommend that someone with NTS expertise who is also familiar with facilitating group discussions be assigned the role of game leader to ensure the success of the session.

Enjoy playing NTS Bingo.

NTS, non-technical skills; ANTSdk, Anaesthesia Non-Technical Skills System – Danish Version; CRM, crisis resource management