0
Original Research: CRITICAL CARE MEDICINE |

Achieving Housestaff Competence in Emergency Airway Management Using Scenario Based Simulation Training*: Comparison of Attending vs Housestaff Trainers FREE TO VIEW

Marnie E. Rosenthal, DO, MPH; Mari Adachi, MD; Vanessa Ribaudo, MD; J. Tristan Mueck, DO; Roslyn F. Schneider, MD, FCCP; Paul H. Mayo, MD, FCCP
Author and Funding Information

*From the Department of Medicine (Drs. Rosenthal, Mueck, and Schneider), Division of Pulmonary and Critical Care Medicine (Drs. Adachi, Ribaudo, and Mayo), Beth Israel Medical Center, New York, NY.

Correspondence to: Paul H. Mayo, MD, FCCP, Division of Pulmonary and Critical Care Medicine, Beth Israel Medical Center, First Ave and Sixteenth St, New York, NY 10003; e-mail: pmayo@chpnet.org



Chest. 2006;129(6):1453-1458. doi:10.1378/chest.129.6.1453
Text Size: A A A
Published online

Study objectives: To evaluate a teaching protocol comparing a critical care attending to a housestaff team in training medical interns in initial airway management skills using a computer-controlled patient simulator (CPS) and scenario-based simulation training (SST).

Design: Prospective, randomized, controlled, unblinded trial.

Setting: Internal medicine residency training program in an urban teaching hospital.

Participants: Forty-nine starting internal medicine interns in July 2003, all of whom had been certified in advanced cardiac life support in June 2003.

Interventions: All interns were tested and scored with a CPS while responding to a standardized respiratory arrest scenario. Random allocation to either training by a single experienced teaching attending or by a housestaff team occurred immediately following testing. All interns were retested using the same scenario 6 weeks following the initial training, and their clinical performance of airway management was scored during actual patient events throughout the year.

Measurements: Initial airway management was divided into specific scorable steps. For each intern, individual step scores and total scores were recorded before and after training. For 10 consecutive months following training, intern airway management scores were recorded for actual patient airway events.

Results: All starting medical interns demonstrated poor initial airway management skills. SST was effective in improving these skills, both on retesting with the patient simulator and in actual patient situations. Interns trained by a housestaff team performed as well as interns trained by the attending.

Conclusions: SST is effective in training medical interns, and the results are equivalent whether the training is provided by an experienced teaching attending or by a housestaff training team.

Patient simulation has emerged as a valuable teaching method in medical education.14 Simulation training offers a controlled, safe, and reproducible environment in which to practice clinical interventions during high-risk, low-frequency events. During events such as cardiopulmonary arrest, a key element of the initial airway management is proper bag-valve mask (BVM) ventilation. Prior to endotracheal intubation by qualified personnel, oxygen must be delivered effectively to the patient’s lungs by BVM, otherwise resuscitation efforts will fail. In a prior study,5 we demonstrated that a computer-controlled patient simulator (CPS) with scenario-based simulation training (SST) combined with intensive one-on-one training by an experienced teaching attending was highly effective in training interns about the proper initial airway management in the apneic patient model, and that these skills were transferable to actual clinical situations. We have now investigated whether an experienced teaching attending was more effective than senior housestaff in training interns in an airway management technique using a CPS and SST. We performed this study to determine whether training skills are unique to an experienced attending or whether teaching skills could be acquired by senior housestaff and to examine whether our previously observed CPS training was reproducible in a new group of interns.

Subjects and Study Design

This study was approved by the Committee on Scientific Activities of Beth Israel Medical Center, NY, and all housestaff consented to participate. All 49 first-year internal medicine house officers beginning on July 1, 2003, were enrolled into the study after receiving standard advanced cardiac life support (ACLS) training and certification in June 2003. A questionnaire was administered to all interns in order to determine prior airway training. Initial airway management was defined as the necessary steps in oxygenating an apneic patient using proper BVM technique. Endotracheal intubation was not included in the testing or training of interns.

In July 2003, the interns were randomized into two groups. Twenty-five interns were assigned to training by an experienced critical care attending (ie, attending-trained [AT] group), and 24 interns were assigned to training by senior housestaff (ie, house-staff-trained [HT] group). Senior housestaff instructors consisted of a group of second-year and third-year residents, a chief medical resident, and a first-year pulmonary fellow. Each housestaff team member assumed primary teaching responsibility for an individual teaching session on a rotating basis. The senior housestaff instructors were individually trained by the critical care attending in teaching technique. Each team member was videotaped and debriefed by the same teaching attending (P.H.M.) while providing mock instruction in order to improve teaching skill 2 days prior to intern testing and training.

The AT group of interns was tested in emergency airway management skills using the CPS, scored by two observers, and immediately trained by a critical care attending in initial airway management skills. The HT group of interns was identically tested and scored using the CPS, and was immediately trained by a housestaff training team member. In order to evaluate skill retention, the two groups were retested 6 weeks later using the same protocol. Additionally, from September 2003 through June 2004 the attending assessed the performance of medical interns in managing actual clinical airway emergencies using the standard scoring instrument. We used the identical testing, scoring, training, and clinical performance assessment as in our previous study.5 We briefly describe the experimental procedure in the next section.

Testing Procedure

The CPS unit (SimMan; Laerdal Medical Corporation; Wappingers Falls, NY) is a human-sized mannequin with realistic features, such as palpable pulses, chest wall excursion, and audible breath sounds. Vital signs are displayed on a bedside monitor and may be assessed by physical examination. The adequacy of BVM ventilation is sensed and graphically represented on a computer screen. The computer operator can program clinical scenarios and record the response of the subject for later analysis.

The testing of interns was standardized, and all testing and training was performed in a typical hospital room that was dedicated to simulation training. The computer was concealed and operated from behind a curtain. The testing team consisted of a computer operator and two researchers who acted as floor nurses. These “nurses” maintained a supportive environment by acting within standard nursing practice but could not prompt or lead the study subject with suggestion or equipment.

Each of the 49 interns was individually escorted to the training room by either the attending or a senior housestaff team member. During a standardized introduction, the trainer demonstrated the capability of the mannequin while the intern physically manipulated the CPS to demonstrate understanding. The intern was instructed that all of the equipment on a typical hospital floor and two nurses were available for assistance. After this introduction, the intern waited in the hallway outside the room to be promptly called to the bedside of the simulated patient as the first responder in an emergency situation.

As the intern stepped out of the room, the CPS was set to apnea with an oxygen saturation of 80%, a BP of 80/60 mm Hg, and a heart rate of 80 beats/min in sinus rhythm. A pulse oximeter and standard hospital code cart with a BVM, and oral airway and intubation tray was positioned out of sight, but was immediately available on request. This scenario mimics a respiratory arrest that had not yet progressed to cardiac arrest. The intern was called into the room by a concerned nurse and was informed that the patient was unresponsive. If successful BVM ventilation did not occur, the oxygen saturation declined followed by bradycardia and progressive hypotension. Full cardiac arrest occurred at 3 min. Successful BVM ventilation resulted in improvements in oxygenation, BP, and heart rate.

Scoring

Immediately after testing, the two nurses independently completed a standardized scoring sheet based on their own observations and the data recorded by the computer operator. Scores of seven predefined essential items of initial airway management and four nonessential but desirable elements were recorded (Table 1 ). The omission of any of the seven essential items resulted in continued oxygen desaturation and cardiac arrest. The omission of any of the four nonessential items permitted successful resuscitation during the scenario.

Training

After testing, the intern was immediately trained using the same simulator and scenario. These sessions were a stepwise debriefing of test performance in which the physiology of apnea and the consequences of respiratory arrest were reviewed in detail. The intern was then trained in the 11 steps using a hands-on approach followed by a second repetition of the scenario. The intern was again intensively debriefed, which was followed by another full run of the scenario and a final comprehensive debriefing. Training sessions typically lasted between 30 and 40 min, and emphasized active task repetition and team communication skills. No intern was dismissed from training without a demonstration of perfect initial airway management skills.

Clinical Performance Assessment

As part of a medical ICU quality improvement program, the attending investigator assessed the performances of medical interns during airway emergencies throughout the year. Initial airway management skills were assessed during full cardiac arrest situations and in the medical ICU during emergency intubation sequences. Interns generally worked in pairs to provide airway support. In order to provide appropriate patient care if quality issues were observed, the attending immediately intervened.

Statistical Analysis

This study was designed to assess the primary end point of whether housestaff training was as effective as that with an experienced teaching attending in imparting the skills necessary for initial airway management. Our initial assumption was that the interns trained by a teaching attending would significantly differ in outcome compared with interns trained by housestaff.

The data were analyzed with the χ2 method applied to 2 × 2 contingency tables. A significance level of 0.05 was used throughout. For within-group analysis, the McNemar test of repeated measures was used. All data were analyzed using a statistical software package (SAS, version 8.0; SAS Institute; Cary, NC).

All 49 interns consented to participate in the study, and were tested and trained per protocol. No significant difference existed in previous airway management experience between the two groups. There were no statistically significant differences in scores before training between the AT and HT groups, with the exception of step 11 (positions intubation tray) [AT group < HT group, p ≤ 0.025]. There were no significant differences in scores following training between the AT and HT groups. After training, the AT group significantly improved their performance in all steps, with the exception of step 8 (attaches pulse oximeter) [Table 2] . The HT group significantly improved their performance in all steps with the exception of step 1 (identifies apnea) and step 8 (attaches pulse oximeter) [Table 2].

During the 10 months following airway training, intern performance of the essential steps of initial airway management was excellent in actual clinical events (Table 3 ). The following two types of clinical situations were scored: BVM ventilation/oxygenation during emergent intubation of critically ill patients without cardiac arrest; and BVM ventilation/oxygenation during emergent intubation of critically ill patients during full cardiac arrest. The attending was able to score 79 initial airway management events performed by a total of 42 different interns. All 42 of the observed interns were assessed one or more times, and all observed encounters involved interns working in pairs. Generally, two interns were involved in each critical airway event; therefore, many interns were observed on more than one occasion. To ensure patient safety, any missed steps were immediately corrected by other team members.

This study shows that SST combined with a CPS is a useful means of training interns in initial airway management, and that an housestaff training team is as effective as an experienced teaching attending in teaching this important skill. The SST improved intern performance on the CPS 6 weeks after initial training and was associated with excellent clinical performance. These results confirm those from our previous study, which was of similar design and had been performed with another intern group 1 year earlier. The training method is reproducible, and the skill required for debriefing can be easily acquired by medical housestaff.

Untrained interns identified apnea and requested a pulse oximeter when faced with the respiratory arrest scenario, but were otherwise unable to organize a response that would save a patient from inevitable hypoxemia and death. No untrained intern was able to perform all of the essential steps required for initial airway management despite recent certification in ACLS. An occasional intern completed one or two essential steps, which represented an inadequate overall response. After training, interns in general had excellent performance in initial airway management when tested on the CPS. In actual clinical situations, the performance rate in essential and nonessential tasks was near perfect; however, the occasional omission underlines the fact that it is difficult to achieve perfect training when dealing with stressful high-risk, low-frequency events.

In our previous study, we identified step 4 (connection of the BVM to the oxygen source) as problematic. Because of this, we emphasized this critical step in training the interns and found an improvement in outcome compared to our earlier study. An SST approach allowed us to break down a complex task into individual scorable steps, to identify a specific problem area, and to improve teaching strategy.

This study has methodological limitations. The scoring of the interns by the nurses, although standardized, could be biased. The complexity of interpersonal interaction between the nurse actors and the interns could not be standardized, nor could the trainer be expected to have the same approach with every intern. In our previous study, we reviewed the video record to verify the scoring performed by the nurse team. We found a single camera angle frequently missed critical elements and did not change the observed score. For this reason, we did not use video scoring in the present study. We acknowledge that multiple camera angles and blinded scoring may have methodological advantages.

We found it difficult to design an evaluation method for clinical training effect. Ideally, an independent observer would have scored all interns at every airway event. Videotaping urgent intubations and cardiac arrests for later scoring was not feasible, due to the abrupt nature of urgent intubations and cardiac arrests.

The training attending performed all clinical scoring and may have been biased toward a favorable outcome. No intern performed clinical initial airway management unassisted, so excellent performance may have been attributed to working as a team. The first clinical event following simulation training would be the most important to score, as repeated exposure to airway management may contribute to improved performance. We conclude that SST training is associated with good clinical performance, but definitive proof requires a stronger study design.

In the clinical assessment phase, this study had no control group. Ideally, we would have trained half of the intern group and compared clinical performance to an untrained control group. Based on the results of our previous work, we thought that this would be unethical. Additionally, interns are able to share clinical skills throughout the year, which may wash out the effect in the control group. Before starting our airway management training program in 2002, we assessed initial airway management during cardiac arrests for a 6-month period. We attempted to score interns’ airway skills at these events but found it difficult, as they were never involved themselves in initial airway management and uniformly deferred to others in this responsibility. Interns were neither assigned the responsibility nor had received training. We surmise that prior to our training program interns lacked competence in initial airway management.

We were unable to compare the actual clinical performance of the AT group with that of the HT group. Interns from either group were frequently paired together during the clinical assessment phase of the study. However, excellent overall performance suggests that the training had similar positive effects on both groups of interns, as it did with retesting on the CPS.

Simulation training programs are often organized as large, independent, highly equipped training centers with dedicated full-time technical staff. Such centers are highly effective in their training effort but require large financial investments for start-up and maintenance costs. An alternative approach may be found in individual academic and community hospitals that have purchased a capable high-fidelity CPS, but lack the financial resource to develop a large high-cost simulation program. A major disadvantage to this approach is the lack of funding for full-time personnel. In this study, we describe a viable model that empowers housestaff to actively participate as instructors, actors, and technical support personnel in an important teaching and quality-improvement program. We emphasize that the housestaff who provided teaching had been specifically trained in this skill by the attending. We cannot comment as to whether housestaff without training in teaching methods would be effective in training other housestaff. Another important question is how often housestaff require retraining in initial airway management. We are not able to answer this question with the present data set, but it will be the subject of a future report.

One advantage of a small hospital-based SST program is that the training method may be used to solve specific problems identified by clinical staff. Our study is an example of such a targeted problem-based approach. We identified a deficiency among medical housestaff in a critical skill, analyzed the process, and developed an SST training method to correct the problem. Another advantage of a limited, hospital-based program is that the training personnel work closely with the housestaff they train. This is a strong motivator for the training team and encourages a personalized approach for the trainee.

From a methodological point of view, we were able to score intern performance easily by identifying discrete tasks that occurred linearly during the scenario. This facilitated the measurement of training outcome. Marsch et al6used a similar approach in determining the adherence to algorithms of first-responders in simulated cardiac arrests using sophisticated scoring methods. A task-oriented approach to SST allows for the identification of problem areas and the assessment of actual clinical performance. One problem encountered in evaluating the efficacy of SST is the difficulty of measuring outcomes. Many studies711 have reported outcomes based on participants’ opinion of training technique. Participant opinion of the training may not actually correlate with successful teaching.12Designing the training sequence in a way that permits the scoring of discrete tasks allows an alternative approach to measuring efficacy.13

In conclusion, we have demonstrated that SST combined with a CPS is effective in providing initial airway management training to medical interns. An housestaff teaching team was as effective as an experienced teaching attending in performing this task. Despite recent ACLS certification, medical interns had very poor initial airway management skills, a result that duplicated the findings of our previous study.5 Initial airway management skills are critical elements of the ACLS protocol; our results suggest that medical housestaff should receive additional training beyond that of a standard ACLS course.

Abbreviations: ACLS = advanced cardiac life support; AT = attending-trained; BVM = bag-valve mask; CPS = computer-controlled patient simulator; HT = housestaff-trained; SST = scenario-based simulation training

No author received any financial support from any organization with financial interests in the subject matter of this article. This research was supported by the Alan and Barbara Mirken Fund.

Table Graphic Jump Location
Table 1. Essential and Nonessential Steps of Initial Airway Management*
* 

Steps 1 to 7 are essential.

Table Graphic Jump Location
Table 2. Intern Performance of Individual Airway Management Steps: AT vs HT Groups*
* 

Values are given as No. (%), unless otherwise indicated. NS = not significant.

 

Prestest vs posttest.

Table Graphic Jump Location
Table 3. Intern Performance of Essential Steps of Airway Management in Clinical Situations*
* 

NA = not applicable. Values are given as No. (%).

Gaba, DM (1992) Improving anesthesiologists’ performance by simulating reality.Anesthesiology76,491-494. [CrossRef] [PubMed]
 
Cavanaugh, S Computerized simulation technology for clinical teaching and testing.Acad Emerg Med1997;10,939-943
 
Issenberg, SB, McGaghie, WC, Hart, IR, et al Simulation technology for health care professional skills training and assessment.JAMA1999;282,861-866. [CrossRef] [PubMed]
 
Friedrich, MJ Practice makes perfect risk-free medical training with patient simulators.JAMA2002;288,2808-2812. [CrossRef] [PubMed]
 
Mayo, PH, Hackney, JE, Mueck, JT, et al Achieving housestaff competence in emergency airway management: results of a teaching program using a computerized patient simulator.Crit Care Med2004;32,2422-2427. [CrossRef] [PubMed]
 
Marsch, S, Tschan, F, Semmer, N, et al Performance of first responders in simulated cardiac arrests.Crit Care Med2005;33,963-967. [CrossRef] [PubMed]
 
Lighthall, GK, Barr, J, Howard, SK, et al Use of a fully simulated intensive care unit environment for critical event management training for internal medicine residents.Crit Care Med2003;31,2437-2443. [CrossRef] [PubMed]
 
Holzman, RS, Cooper, JB, Gaba, DM, et al Anesthesia crisis resource management: real-life simulation training in operating room crises.J Clin Anesth1995;7,675-687. [CrossRef] [PubMed]
 
Hammond, J, Bermann, M, Chen, B, et al Incorporation of a computerized human patient simulator in critical care training: a preliminary report.J Trauma2002;53,1064-1067. [CrossRef] [PubMed]
 
Owen, H, Plummer, JL Improving learning of a clinical skill: the first year’s experience of teaching endotracheal intubation in a clinical simulation facility.Med Educ2002;36,635-642. [CrossRef] [PubMed]
 
Small, SD, Wuerz, RC, Simon, R, et al Demonstration of high-fidelity simulation team training for emergency medicine.Acad Emerg Med1999;6,312-323. [CrossRef] [PubMed]
 
France, DJ, Stiles, R, Gaffney, FA, et al Crew resource management training: clinicians’ reactions and attitudes.AORN J2005;82,214-224
 
Chopra, V, Gesink, BJ, de Jong, J, et al Does training on an anaesthesia simulator lead to improvement in performance?Br J Anaesth1994;73,293-297. [CrossRef] [PubMed]
 

Figures

Tables

Table Graphic Jump Location
Table 1. Essential and Nonessential Steps of Initial Airway Management*
* 

Steps 1 to 7 are essential.

Table Graphic Jump Location
Table 2. Intern Performance of Individual Airway Management Steps: AT vs HT Groups*
* 

Values are given as No. (%), unless otherwise indicated. NS = not significant.

 

Prestest vs posttest.

Table Graphic Jump Location
Table 3. Intern Performance of Essential Steps of Airway Management in Clinical Situations*
* 

NA = not applicable. Values are given as No. (%).

References

Gaba, DM (1992) Improving anesthesiologists’ performance by simulating reality.Anesthesiology76,491-494. [CrossRef] [PubMed]
 
Cavanaugh, S Computerized simulation technology for clinical teaching and testing.Acad Emerg Med1997;10,939-943
 
Issenberg, SB, McGaghie, WC, Hart, IR, et al Simulation technology for health care professional skills training and assessment.JAMA1999;282,861-866. [CrossRef] [PubMed]
 
Friedrich, MJ Practice makes perfect risk-free medical training with patient simulators.JAMA2002;288,2808-2812. [CrossRef] [PubMed]
 
Mayo, PH, Hackney, JE, Mueck, JT, et al Achieving housestaff competence in emergency airway management: results of a teaching program using a computerized patient simulator.Crit Care Med2004;32,2422-2427. [CrossRef] [PubMed]
 
Marsch, S, Tschan, F, Semmer, N, et al Performance of first responders in simulated cardiac arrests.Crit Care Med2005;33,963-967. [CrossRef] [PubMed]
 
Lighthall, GK, Barr, J, Howard, SK, et al Use of a fully simulated intensive care unit environment for critical event management training for internal medicine residents.Crit Care Med2003;31,2437-2443. [CrossRef] [PubMed]
 
Holzman, RS, Cooper, JB, Gaba, DM, et al Anesthesia crisis resource management: real-life simulation training in operating room crises.J Clin Anesth1995;7,675-687. [CrossRef] [PubMed]
 
Hammond, J, Bermann, M, Chen, B, et al Incorporation of a computerized human patient simulator in critical care training: a preliminary report.J Trauma2002;53,1064-1067. [CrossRef] [PubMed]
 
Owen, H, Plummer, JL Improving learning of a clinical skill: the first year’s experience of teaching endotracheal intubation in a clinical simulation facility.Med Educ2002;36,635-642. [CrossRef] [PubMed]
 
Small, SD, Wuerz, RC, Simon, R, et al Demonstration of high-fidelity simulation team training for emergency medicine.Acad Emerg Med1999;6,312-323. [CrossRef] [PubMed]
 
France, DJ, Stiles, R, Gaffney, FA, et al Crew resource management training: clinicians’ reactions and attitudes.AORN J2005;82,214-224
 
Chopra, V, Gesink, BJ, de Jong, J, et al Does training on an anaesthesia simulator lead to improvement in performance?Br J Anaesth1994;73,293-297. [CrossRef] [PubMed]
 
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

CHEST Journal Articles
  • CHEST Journal
    Print ISSN: 0012-3692
    Online ISSN: 1931-3543