0
CME: ACCP Evidence-Based Educational Guidelines |

The Role of Audience Characteristics and External Factors in Continuing Medical Education and Physician Change: Effectiveness of Continuing Medical Education: American College of Chest Physicians Evidence-Based Educational Guidelines FREE TO VIEW

Mary Martin Lowe, PhD; Nancy Bennett, PhD; Alejandro Aparicio, MD
Author and Funding Information

*From the Accreditation Council for Continuing Medical Education (Dr. Lowe), Chicago, IL; Harvard Medical School-Massachusetts General Hospital (Dr. Bennett), Boston, MA; and the American Medical Association (Dr. Aparicio), Chicago, IL.

Correspondence to: Mary Martin Lowe, PhD, Accreditation Council for Continuing Medical Education, 515 N State St, Ste 1801, Chicago, IL 60654; e-mail: Mlowe@accme.org


Reproduction of this article is prohibited without written permission from the American College of Chest Physicians (www.chestjournal.org/misc/reprints.shtml).


Chest. 2009;135(3_suppl):56S-61S. doi:10.1378/chest.08-2519
Text Size: A A A
Published online

Background:  The Agency for Healthcare Research and Quality (AHRQ) Evidence Report identified and assessed audience characteristics (internal factors) and external factors that influence the effectiveness of continuing medical education (CME) in changing physician behavior.

Methods:  Thirteen studies examined a series of CME audience characteristics (internal factors), and six studies looked at external factors to reinforce the effects of CME in changing behavior.

Results:  With regard to CME audience characteristics, the 13 studies examined age, gender, practice setting, years in practice, specialty, foreign vs US medical graduate, country of practice, personal motivation, nonmonetary rewards and motivations, learning satisfaction, and knowledge enhancement. With regard to the external characteristics, the six studies looked at the role of regulation, state licensing boards, professional boards, hospital credentialing, external audits, monetary and financial rewards, academic advancement, provision of tools, public demand and expectations, and CME credit. No consistent findings were identified.

Conclusions:  The AHRQ Evidence Report provides no conclusions about the ways that internal or external factors influence CME effectiveness in changing physician behavior. However, given what is known about how individuals approach learning, it is likely that internal factors play an important role in the design of effective CME. Regulatory and professional organizations are providing new structures, mandates, and recommendations for CME activities that influence the way CME providers design and present activities, supporting a role that is not yet clear for external factors. More research is needed to understand the impact of these factors in enhancing the effectiveness of CME.

Organizations providing continuing medical education (CME) must develop activities that integrate important content, strategies for supporting learning, and the requirements of standard-setting organizations. Knowing more about the ways that audience characteristics (internal factors) and regulatory and licensing bodies (external factors) influence best practice supports providers in making better choices for more effective learning that will change behavior. At the same time, knowing more about the impact of these factors helps physicians to make better choices as they structure their learning.

How does practice experience shape the questions raised by a group of learners participating in CME activities? Does providing CME credit influence what participants learn? Although these questions are important, we have little information to understand not only the role of individual differences in learning, but also the ways that regulatory groups influence the way providers develop CME activities. This chapter presents a summary of the Agency for Healthcare Research and Quality (AHRQ) Evidence Report1 about audience characteristics (internal) and external factors that influence the effectiveness of CME, outlines ideas for providers about research findings in other fields that may increase CME effectiveness, and suggests future research.

The AHRQ Evidence Report formulated specific questions followed by a systematic literature search using eligibility criteria. Of the 136 studies cited in the report, 13 studies214 addressed the question of the influence of audience characteristics (internal factors) on educational interventions, and 6 studies8,1418 addressed the influence of external factors in reinforcing the effects of CME on changing behavior.

Which characteristics of the audience by themselves or in combination with other characteristics influence the effectiveness of specific educational techniques? Characteristics of the audience considered were age, gender, practice setting, years in practice, specialty, foreign vs US medical graduate, country of practice, personal motivation, nonmonetary rewards and motivations, learning satisfaction, and knowledge enhancement. The question posed was whether subgroups of an audience would respond differently to an educational intervention and which educational techniques would be most effective for a given subgroup.

The 13 articles214 that met study criteria examined audience characteristics that influence educational interventions of CME activities. Six articles2,4,8,9,10,13 addressed years in practice, and six3,8,9,11,12,14 addressed the role of age; none reported significant findings. No major findings were noted consistently in studies about gender3,4,8,9,14 or race,8 and there was no association between desired outcome of an activity and board certification.4,8,12 Practice setting8,12 and specialty58,12 were linked to minor findings that were not considered significant. No differences in educational outcomes were found between US and foreign medical school graduates8 or those with or without residency training.8 No reports about personal motivation met study criteria. Greater experience may be a factor in improved attitude2 or self-reported practice behavior.2 None of the studies demonstrated a link between years in practice and new knowledge, skills, or practice outcomes. Except for one study,12 audience characteristics were secondary to other research goals.

A few reports provided minor, nonsignificant findings. Of the five studies related to gender, one9 showed a greater increase in women's confidence compared to men's confidence in the performance of knee joint injections after a CME intervention using printed materials, hands-on instruction, or video instruction. The effectiveness of cue enhancement on mammography screening rates8 was higher for nonwhites than for whites. The same study showed a greater effect of cue enhancement on solo practitioners compared to those in other practice settings.

Which external factors by themselves or in combination with other factors reinforce the effects of CME in changing behavior? Factors that were examined by the AHRQ Evidence Report included the role of regulation, state licensing boards, professional boards, hospital credentialing, external audits, monetary and financial rewards, academic advancement, provision of tools, public demand and expectations, and CME credit. Commitment-to-change statements were categorized as an external influence because they viewed a physician's commitment to change as creating an external expectation or motivating factor to change.

Using the AHRQ search criteria, very little research about the reinforcing effects of external factors in changing behavior was identified. Based on six articles,8,1418 there were limited findings about the role of external factors alone or in combination with other factors. Three studies1517 looked at CME credit as a factor, with no significant findings. Grady et al8 examined monetary rewards to enhance educational outcomes, finding no added benefit. Commitment-to-change statements, included in two studies14,18 that looked at age and gender, were found to have some impact on the effectiveness of CME. In these studies,14,18 participants' expression of intent to change was more likely to result in a change in practice behavior, with or without a participant signature.

There was not enough evidence to answer the following two AHRQ Evidence Report questions: “Which characteristics of the audience by themselves or in combination with other characteristics influence the effectiveness of certain educational techniques?” and “Which external factors by themselves or in combination with other factors reinforce the effects of CME in changing behavior?” The lack of findings that would shape conclusions points to the need for research about the impact of internal and external characteristics on CME effectiveness.

The AHRQ Evidence Report created questions and set specific search criteria using the methodology of clinical evidence-based medicine. The results were shaped by the strengths and weaknesses of this method and were likely influenced by looking for educationally based research employing an approach traditionally used for quantitative clinical research. By comparison, in two other major translational reviews19,20 that did not set specific search criteria, authors were able to select a wide variety of sources, which contributed to a deeper understanding of CME. In another approach intended to standardize and compare work,2123 the analysis was limited to published randomized controlled trials. These findings allowed more direct comparison but left out some of the complexity of learning, change, and educational interventions that does not lend itself to this kind of quantitative analysis. Each source provided some similar findings and some differing conclusions. Based on studies not included in the AHRQ report,2123 more effective CME learning activities include a clear identification of need or practice gap, interactivity and feedback, and ongoing learning or multiple sessions. Those ideas point to design rather than learner characteristics. However, it is possible to look at the impact on individuals using those criteria. Clear identification of need provides each learner with a starting point to examine a comparison of personal need within a group. Interactivity and feedback provide a mechanism for an individual to test out ideas. No study results that test the impact of these criteria on individuals were found, and this would be an interesting area for future research.

Although these reports and others offer strategies that have proven to be effective in supporting physician learning and change, providers of CME are still left with questions about how individual or group characteristics may affect a strategy's effectiveness and how external factors relate to the learning process. Although we work from a rich, developing literature base, the complexity of the questions necessary to understand physicians' learning processes has led authors to approach queries with many different methodologies, making comparisons difficult. Further, the lack of standardized terminology and the variability in the choice of search terms contribute to the kinds of research selected to interpret, often leaving out significant work. Major changes in the healthcare field and in CME raise questions about the relevance of some of the earlier CME research that may have been based on assumptions that no longer apply. These are some of the limitations of the AHRQ Evidence Report. Studies not meeting the criteria for this review may discuss meaningful issues regarding effective CME. Importantly, CME activities are not usually limited to a specific audience by gender, age, or other similar factors, which may not allow conclusions about specific audience characteristics. Regulatory forces may prohibit or limit manipulation of some external factors that influence behavior changes.

Research on Internal Factors

The diverse groups of physicians who participate in CME present questions for providers and researchers. Although age was not found to be a factor that has an impact on the effectiveness of CME in the current report, it seems plausible that physician's participation in more technologically sophisticated CME activities might vary by age. As technology continues to influence CME planning, how will providers support physicians' differing approaches to learning? How does gender, ethnicity, or cultural background influence the effectiveness of CME? Although some differences were found in two articles,8,9 more research is needed to explore these results and provide an understanding of why such differences exist. As CME responds to the changes in the demographics of physicians, findings may change.

In the AHRQ Evidence Report, the ways that years in practice influence learning produced mixed results. Simply looking at age and years of experience does not predict expert performance. However, studies2426 show that time spent in deliberate practice activities is a predictor of expert performance. Another facet of performance not often a part of CME research is an individual's mental model. Literature on mental models2729 offers insight into the link between performance and perspective or the framework that each individual uses to think about the way he or she approaches patient care. By considering a physician's mental model for diagnosis and treatment, we gather rich data about gaps in practice for planning a CME activity. If a physician does not “think” that talking to patients about smoking cessation is effective in changing behavior, then the physician will see no reason to have conversations with patients about the need to stop smoking. In this way, a physicians' mental model plays a key role in his or her learning and change process, influencing the effectiveness of CME.

The AHRQ Evidence Report found no CME sources that evaluated the impact of internal or personal motivation factors. Extending that search to the fields of human resource development and psychology accesses a rich literature base and theories to help develop integrative models of multiple internal factors and their relationships with performance. For example, an extensive metaanalysis30 of more than 20 years of research on training motivation resulted in a comprehensive pathway for transfer of learning to performance. This pathway included 14 variables, the relationships among the variables, and how the variables affect transfer and performance. Some of the variables included were achievement motivation, self-assessment, self-efficacy, organizational commitment, and career commitment. This body of evidence certainly could be used for hypothesis generation in CME research.

Research on External Factors

There is limited research on how external factors affect CME. One of the problems in defining the impact of external factors is that many variables cannot be manipulated because of requirements for licensure, professional membership, or hospital privileges. Designing a study about external influences is difficult to structure because of the barriers related to incentives, which may be professionally inappropriate or banned by regulation. In addition, there is no common definition of what constitutes an external factor. Requirements mandated by law or for membership in a professional organization may be considered as external factors. However, when teaching techniques that may be related to retention of content from a CME activity, such as commitment to change, are considered, there may be debate about whether those are part of the teaching-and-learning process or may be thought of as external factors. Despite the lack of clarity on whether the commitment- to-change technique is truly an external factor, there is sound research, both from a randomized controlled study31 and from a review of research studies,32 that demonstrates the value of the commitment-to-change tool in facilitating physician learning and behavior change. Further research and discussion on this technique are warranted.

Literature from human resource development and psychology contribute to the analysis of external factors not yet explored in CME. For example, organizational climate as an external factor could lead to intriguing CME research on the impact of the climate of a physician's medical specialty, region, or state. Researchers might pose such questions as, “Do high medical malpractice rates in specialties such as neurosurgery or obstetrics influence physicians' motivation to learn?” Peer support and organizational support are among factors studied in other fields that also could be explored in CME. For example, what is the role of support from a hospital or peers in a group practice in terms of motivation to learn?

No link between certification and CME effectiveness was found in the AHRQ Evidence Report. Changes in the field warrant a new look in this direction. In the United States, as more specialties implement Maintenance of Certification programs for individual physicians, findings from new research will be critical. In addition, in this report there was no link between providing CME credit and changes in performance. Changes in Maintenance of Certification programs that expect physicians to demonstrate lifelong learning and periodic self-assessment may provide new clues to effective personalized systems.

Another important facet of CME research is the external motivating factors that affect organizations that plan CME. For example, to what extent do CME regulation and accreditation influence a CME provider's choice of educational methodologies? Groups in the United States, such as the American Academy of Family Physicians, the Accreditation Council on Continuing Medical Education, and the American Medical Association, set specific mandates that are important to the daily work of CME providers and that influence requirements for participants. The degree to which these kinds of external factors affect individual physicians and organizations and personal motivating factors will work synergistically to affect the effectiveness of CME is not clear.

The theory base in CME is broad, with many different ideas that shape the way we think about how physicians learn. We are in the early stages of work to understand a very complex topic. Although the AHRQ Evidence Report provided no substantive findings about the influence of internal or external factors on the effectiveness of CME in changing behavior, it represents a type of work that is needed to bring greater understanding to how physicians learn and change. Through projects like the AHRQ Evidence Report, researchers have a different view into the gaps in our understanding of effective CME. This highlights one of these gaps: The literature included in the AHRQ Evidence Report does not present a complete picture of how audience characteristics or external factors affect the effectiveness of CME in changing behavior.

The selected studies about internal factors that influence CME did not provide clear direction on how to enhance CME for a particular subgroup of participants. It is not feasible to limit most CME activities according to audience characteristics, but with more research and a greater understanding of the impact of individual characteristics on learning, we may be able to be more specific in the objectives or goals of activities to help learners effectively select activities. External factors that influence CME are important to both learners and CME providers. The drive by regulatory and professional groups to set standards, provide guidelines, and point to goals for physician performance affects the way that providers do their work. Major changes in healthcare and in CME point to the need for continued research to support CME providers as they help physicians to learn. To that end, providers and researchers must uncover answers about the internal and external factors in effective learning with the goal of behavior changes that lead to better patient care.

AHRQ

Agency for Healthcare Quality and Research

CME

Continuing Medical Education

Dr. Lowe is the Director of Accreditation and Recognition Services for the Accreditation Council for Continuing Medical Education.

Dr. Bennett has served as a consultant with Reed Medical Education about CME, receiving dollar amounts under $2,000.

Dr. Aparicio has received grant monies from the National Task Force on CME and has been a speaker for the American Medical Association on topics related to this article.

Marinopoulos SS, Dorman T, Ratanawongsa N, et al. Effectiveness of continuing medical education. 2007; Rockville, MD Agency for Healthcare Research and Quality evidence report/technology assessment No. 149.
 
Kiang KM, Kieke BA, Como-Sabetti K, et al. Clinician knowledge and beliefs after statewide program to promote appropriate antimicrobial drug use. Emerg Infect Dis. 2005;11:904-911. [PubMed] [CrossRef]
 
Harris JMJ, Kutob RM, Surprenant ZJ, et al. Can Internet-based education improve physician confidence in dealing with domestic violence? Fam Med. 2002;34:287-292. [PubMed]
 
Gerstein HC, Reddy SS, Dawson KG, et al. A controlled evaluation of a national continuing medical education programme designed to improve family physicians' implementation of diabetes specific clinical practice guidelines. Diabet Med. 1999;16:964-969. [PubMed]
 
Des Marchais JE, Jean P, Castonguay LG. Training psychiatrists and family doctors in evaluating interpersonal skills. Med Educ. 1990;24:376-381. [PubMed]
 
Lane DS, Polednak AP, Burg MA. Effect of continuing medical education and cost reduction on physician compliance with mammography screening guidelines. J Fam Pract. 1991;33:359-368. [PubMed]
 
Lewis CE, Bursch B, Klau M, et al. Continuing medical education for AIDS: an organizational response. AIDS Educ Prev. 1993;5:263-271. [PubMed]
 
Grady KE, Lemkau JP, Lee NR, et al. Enhancing mammography referral in primary care. Prev Med. 1997;26:791-800. [PubMed]
 
Leopold SS, Morgan H, Kadel NJ, et al. Impact of educational intervention on confidence and competence in the performance of a simple surgical task. J Bone Joint Surg Am. 2005;87:1031-1037. [PubMed]
 
Juzych NS, Banerjee M, Essenmacher L, Lerner SA. Improvements in antimicrobial prescribing for treatment of upper respiratory tract infections through provider education. J Gen Intern Med. 2005;20:901-905. [PubMed]
 
Davis RS, Bukstein DA, Luskin AT, et al. Changing physician prescribing patterns through problem-based learning: an interactive, teleconference case-based education program and review of problem based learning. Ann Allergy Asthma Immunol. 2004;93:237-242. [PubMed]
 
Soumerai SB, Avorn J. Predictors of physician prescribing change in an educational experiment to improve medication use. Med Care. 1987;25:210-221. [PubMed]
 
Beaulieu MD, Rivard M, Hudon E, et al. Comparative trial of a short workshop designed to enhance appropriate use of screening tests by family physicians. Can Med Assoc J. 2002;167:1241-1246
 
Mazmanian PE, Johnson RE, Zhang A, et al. Effects of a signature on rates of change: a randomized controlled trial involving continuing education and the commitment-to-change model. Acad Med. 2001;76:642-646. [PubMed]
 
Chassin MR, McCue SM. A randomized trial of medical quality assurance: improving physicians' use of pelvimetry. JAMA. 1986;256:1012-1016. [PubMed]
 
Messina CR, Lane DS, Grimson R. Effectiveness of women's telephone counseling and physician education to improve mammography screening among women who underuse mammography. Ann Behav Med. 2002;24:278-289
 
Schaafsma F, Hulshof C, van Dijk F, et al. Information demands of occupational health physicians and their attitude towards evidence based medicine. Scand J Work Environ Health. 2004;30:327-330. [PubMed]
 
Pereles L, Lockyer J, Hogan D, et al. Effectiveness of commitment contracts in continuing medical education. Acad Med. 1996;71:394. [PubMed]
 
Davis DA, Fox RD. The physician as learner: linking research to practice. 1994; Chicago, IL American Medical Association
 
Davis D, Barnes BE, Fox R. The Continuing Professional Development of Physicians: From Research to Practice. 2003; Chicago, IL American Medical Association
 
Mazmanian P, Davis D. Continuing medical education and the physician as a learner: guide to the evidence. JAMA. 2002;288:1057-1060. [PubMed]
 
Davis D, Thomson MA, Oxman A, et al. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA. 1999;274:700-705
 
Davis DA, Thomson MA, Oxman AD, et al. Evidence for the effectiveness of CME: a review of 50 randomized controlled trials. JAMA. 1992;268:1111-1117. [PubMed]
 
Charness N, Tuffiash M, Krampe R, et al. The role of deliberate practice in chess expertise. Appl Cogn Psychol. 2005;19:151-165
 
Ericsson KA, Krampe R, Tesch-Romer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev. 1993;100:363-406
 
Krampe RT, Ericsson KA. Maintaining excellence: deliberate practice and elite performance in young and older pianists. J Exp Psychol. 1996;125:331-359
 
Gentner D.Smelser NJ, Baes PB. Psychology of mental models. International encyclopedia of the social and behavioral sciences. 2002; Amsterdam, the Netherlands Elsevier Science
 
Staggers N, Norcio AF. Mental models: concepts for human-computer interaction research. Int J Man Mach Stud. 1993;38:587-605
 
Ellis S, Davidi I. After-event reviews: drawing from successful and failed experiences. J Appl Psychol. 2005;90:857-871. [PubMed]
 
Colquitt JA, LaPine JA, Noe RA. Toward an integrative theory of training motivation: a meta-analytic path analysis of 20 years of research. J Appl Psychol. 2000;85:678-707. [PubMed]
 
Wakefield J, Herbert CP, Maclure M, et al. Commitment to change statements can predict actual change in practice. J Contin Educ Health Prof. 2003;23:81-93. [PubMed]
 
Wakefield J. Commitment to change: exploring its role in changing physician behavior through continuing education. J Contin Educ Health Prof. 2004;24:197-204. [PubMed]
 

Figures

Tables

References

Marinopoulos SS, Dorman T, Ratanawongsa N, et al. Effectiveness of continuing medical education. 2007; Rockville, MD Agency for Healthcare Research and Quality evidence report/technology assessment No. 149.
 
Kiang KM, Kieke BA, Como-Sabetti K, et al. Clinician knowledge and beliefs after statewide program to promote appropriate antimicrobial drug use. Emerg Infect Dis. 2005;11:904-911. [PubMed] [CrossRef]
 
Harris JMJ, Kutob RM, Surprenant ZJ, et al. Can Internet-based education improve physician confidence in dealing with domestic violence? Fam Med. 2002;34:287-292. [PubMed]
 
Gerstein HC, Reddy SS, Dawson KG, et al. A controlled evaluation of a national continuing medical education programme designed to improve family physicians' implementation of diabetes specific clinical practice guidelines. Diabet Med. 1999;16:964-969. [PubMed]
 
Des Marchais JE, Jean P, Castonguay LG. Training psychiatrists and family doctors in evaluating interpersonal skills. Med Educ. 1990;24:376-381. [PubMed]
 
Lane DS, Polednak AP, Burg MA. Effect of continuing medical education and cost reduction on physician compliance with mammography screening guidelines. J Fam Pract. 1991;33:359-368. [PubMed]
 
Lewis CE, Bursch B, Klau M, et al. Continuing medical education for AIDS: an organizational response. AIDS Educ Prev. 1993;5:263-271. [PubMed]
 
Grady KE, Lemkau JP, Lee NR, et al. Enhancing mammography referral in primary care. Prev Med. 1997;26:791-800. [PubMed]
 
Leopold SS, Morgan H, Kadel NJ, et al. Impact of educational intervention on confidence and competence in the performance of a simple surgical task. J Bone Joint Surg Am. 2005;87:1031-1037. [PubMed]
 
Juzych NS, Banerjee M, Essenmacher L, Lerner SA. Improvements in antimicrobial prescribing for treatment of upper respiratory tract infections through provider education. J Gen Intern Med. 2005;20:901-905. [PubMed]
 
Davis RS, Bukstein DA, Luskin AT, et al. Changing physician prescribing patterns through problem-based learning: an interactive, teleconference case-based education program and review of problem based learning. Ann Allergy Asthma Immunol. 2004;93:237-242. [PubMed]
 
Soumerai SB, Avorn J. Predictors of physician prescribing change in an educational experiment to improve medication use. Med Care. 1987;25:210-221. [PubMed]
 
Beaulieu MD, Rivard M, Hudon E, et al. Comparative trial of a short workshop designed to enhance appropriate use of screening tests by family physicians. Can Med Assoc J. 2002;167:1241-1246
 
Mazmanian PE, Johnson RE, Zhang A, et al. Effects of a signature on rates of change: a randomized controlled trial involving continuing education and the commitment-to-change model. Acad Med. 2001;76:642-646. [PubMed]
 
Chassin MR, McCue SM. A randomized trial of medical quality assurance: improving physicians' use of pelvimetry. JAMA. 1986;256:1012-1016. [PubMed]
 
Messina CR, Lane DS, Grimson R. Effectiveness of women's telephone counseling and physician education to improve mammography screening among women who underuse mammography. Ann Behav Med. 2002;24:278-289
 
Schaafsma F, Hulshof C, van Dijk F, et al. Information demands of occupational health physicians and their attitude towards evidence based medicine. Scand J Work Environ Health. 2004;30:327-330. [PubMed]
 
Pereles L, Lockyer J, Hogan D, et al. Effectiveness of commitment contracts in continuing medical education. Acad Med. 1996;71:394. [PubMed]
 
Davis DA, Fox RD. The physician as learner: linking research to practice. 1994; Chicago, IL American Medical Association
 
Davis D, Barnes BE, Fox R. The Continuing Professional Development of Physicians: From Research to Practice. 2003; Chicago, IL American Medical Association
 
Mazmanian P, Davis D. Continuing medical education and the physician as a learner: guide to the evidence. JAMA. 2002;288:1057-1060. [PubMed]
 
Davis D, Thomson MA, Oxman A, et al. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA. 1999;274:700-705
 
Davis DA, Thomson MA, Oxman AD, et al. Evidence for the effectiveness of CME: a review of 50 randomized controlled trials. JAMA. 1992;268:1111-1117. [PubMed]
 
Charness N, Tuffiash M, Krampe R, et al. The role of deliberate practice in chess expertise. Appl Cogn Psychol. 2005;19:151-165
 
Ericsson KA, Krampe R, Tesch-Romer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev. 1993;100:363-406
 
Krampe RT, Ericsson KA. Maintaining excellence: deliberate practice and elite performance in young and older pianists. J Exp Psychol. 1996;125:331-359
 
Gentner D.Smelser NJ, Baes PB. Psychology of mental models. International encyclopedia of the social and behavioral sciences. 2002; Amsterdam, the Netherlands Elsevier Science
 
Staggers N, Norcio AF. Mental models: concepts for human-computer interaction research. Int J Man Mach Stud. 1993;38:587-605
 
Ellis S, Davidi I. After-event reviews: drawing from successful and failed experiences. J Appl Psychol. 2005;90:857-871. [PubMed]
 
Colquitt JA, LaPine JA, Noe RA. Toward an integrative theory of training motivation: a meta-analytic path analysis of 20 years of research. J Appl Psychol. 2000;85:678-707. [PubMed]
 
Wakefield J, Herbert CP, Maclure M, et al. Commitment to change statements can predict actual change in practice. J Contin Educ Health Prof. 2003;23:81-93. [PubMed]
 
Wakefield J. Commitment to change: exploring its role in changing physician behavior through continuing education. J Contin Educ Health Prof. 2004;24:197-204. [PubMed]
 
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Find Similar Articles
CHEST Journal Articles
PubMed Articles
  • CHEST Journal
    Print ISSN: 0012-3692
    Online ISSN: 1931-3543