0
CME: ACCP Evidence-Based Educational Guidelines |

Continuing Medical Education Effect on Clinical Outcomes: Effectiveness of Continuing Medical Education: American College of Chest Physicians Evidence-Based Educational Guidelines FREE TO VIEW

Paul E. Mazmanian, PhD; David A. Davis, MD; Robert Galbraith, MD
Author and Funding Information

*From the Department of Family Medicine (Dr. Mazmanian), Virginia Commonwealth University, Richmond, VA; the Association of American Medical Colleges (Dr. Davis), Washington, DC; and the Center for Innovation (Dr. Galbraith), National Board of Medical Examiners, Philadelphia, PA.

Correspondence to: Paul E. Mazmanian, PhD, Department of Family Medicine, Associate Dean, Continuing Professional Development and Evaluation Studies, School of Medicine, Box 980048, Virginia Commonwealth University, Richmond, VA 23298-0048; e-mail: paul.mazmanian@vcu.edu


Reproduction of this article is prohibited without written permission from the American College of Chest Physicians (www.chestjournal.org/misc/reprints.shtml).


Chest. 2009;135(3_suppl):49S-55S. doi:10.1378/chest.08-2518
Text Size: A A A
Published online

Background:  As opportunities for quality improvement become more visible, educational planners, health services researchers, and policymakers search for strategies that lead to better clinical outcomes. Continuing medical education (CME) is one such strategy, but the impact of CME is poorly defined in relation to clinical outcomes, and efforts to standardize definitions of clinical outcomes are in varied stages of development.

Methods:  The Johns Hopkins University Evidence-based Practice Center conducted a systematic review of the effectiveness of CME for the Agency for Health Care Research and Quality. From the review, 37 studies were used by the guideline panel to answer questions about improvement in clinical outcomes. Recommendations were made using the American College of Chest Physicians guideline grading system.

Results:  Multiple media, multiple techniques of instruction, and multiple exposures to content are suggested to meet instructional objectives intended to improve clinical outcomes.

Conclusions:  There are models to describe and guide the planning and evaluation of CME, and there are models to measure quality of care. Research and practice of CME must be defined in relation to guideline implementation and quality improvement and other interventions and systems intended to improve or measure clinical outcomes. Further research is required to identify the qualities essential for measuring causal linkages thought to exist among CME, physician behavior, and clinical outcomes.

Figures in this Article

  1. General: We suggest that CME activities be used to improve clinical practice outcomes (Grade 2C).

  2. Instructional media: We suggest the use of multiple media compared to single-medium interventions to improve clinical outcomes (Grade 2C).

  3. Instructional techniques: We suggest the use of multiple instructional techniques compared to single instructional techniques to improve clinical outcomes (Grade 2C).

  4. Frequency of exposure: We suggest multiple exposures to content to meet instructional objectives intended to improved clinical outcomes (Grade 2C).

It is said that in an ideal world, every person who is ill would receive the best care every time from every physician. However, many patients receive less than perfect care, and the continuing education of physicians is seen as fundamental to improving the care that is given. Continuing education includes instruction designed to help physicians acquire and apply scientific knowledge, demonstrate skill, and perform effectively as caregivers.1 What happens to patients once care has been rendered is a clinical outcome.2 With increasing frequency, clinical outcomes are recognized as products delivered in systems of care made up of interdependent parts or agents, including caregivers and patients, bound by a common purpose and acting on their knowledge.2

In a 2007 systematic review of continuing medical education (CME), Marinopoulos et al3 asked whether clinical practice outcomes were influenced by CME activities and, if so, whether the effects persisted for ≥ 30 days after completion of the educational activity. Like other reviews of effective practice and organization of care,49 printed educational materials, conferences, outreach visits, academic detailing, local opinion leaders, patient-mediated interventions, audit and feedback, reminders, and specific conditions of interventions (eg, characteristics of learners and the learning environment) were important variables of study (see Table 3 in the Methods article9a). Unlike prior studies, Marinopoulos et al3 categorized CME by media methods, educational techniques, and amount of exposure (Table 1).

Table Graphic Jump Location
Table 1 Summary of CME Activity Characteristics*

* Values are given as No. (%). From Marinopoulos et al3 (Table 4, page 26).

The systematic evaluation of education is largely a development of the 20th century, growing dramatically with post-World War II social programs, affixing tightly to medical education during the 1960s, and continuing to mature with the growth of information technology and statistical methods during the latter decades. Evaluation models popular in medical education are Miller's pyramid,1 which is well recognized for its familiar descriptive categories of assessing whether the medical learner knows, knows how, shows how, or does (see the Effect on Physician Knowledge9b article, Fig 1); Moore's Outcome-based CME Evaluation Model,10 which measures physician participation, satisfaction, learning, performance, patient health, and population health (Table 2); and the Accreditation Council for Continuing Medical Education framework,11 which accounts for physicians' participation in CME as well as gains in knowledge, competence, and performance. There is no single standardized model apparent for evaluating the effects of individual CME activities, and no single standardized model appears to exist for evaluating clinical outcomes in health care.

Figure Jump LinkFigure 1 The conceptual framework of the MOS.Grahic Jump Location
Table Graphic Jump Location
Table 2 Levels of an Outcomes-Based CME Evaluation Model*

* From Moore10 (Table 13-1, page 251).

The range of events considered to be clinical outcomes is notably broad. It might include the blood glucose level in a patient with diabetes, BP in a patient with hypertension, abnormal chest radiograph following treatment for pneumonia, or kidney function after transplantation; it also may include complications of treatment, such as bleeding after colonoscopic biopsy, allergic reaction to an antibiotic or injection from iodinated contrast material, graft occlusion after cardiac bypass surgery, infant mortality following emergency Caesarian delivery, or hospital death rate.12

Outcomes of care also may include health-related quality of life (HRQL), a concept commonly explored in the field of medical outcomes research and one that includes wide variation in the human experience, with studies involving access to the daily necessities of life such as food and shelter, intrapersonal and interpersonal response to life events, and activities associated with professional fulfillment and personal happiness.12 Typically, HRQL data are collected through patient self-administered questionnaires organized into scales measuring an aspect or domain of HRQL. General HRQL instruments typically address components of overall well-being and function in the physical, emotional, and social domains. Disease-targeted HRQL instruments focus on the impact of particular organic dysfunctions or directly relevant domains, such as anxiety about cancer recurrence, dizziness from antihypertension medication, or suicidal thoughts during depression therapy.12

Marinopoulos et al3 defined clinical outcomes as any change in patient health status, health-related behavior of patients, or attitudes of the patients about the physicians toward whom the CME intervention was directed. Thus, in addition to measures of health status, such as BP and fasting blood glucose, measures of patient satisfaction, medication adherence, and smoking cessation were included. These measures are consistent with Miller's general model of medical learner assessment1 and coincide with the highest levels of performance proposed by Moore10 and the Accreditation Council for Continuing Medical Education.11 They also are consistent with descriptions of clinical outcomes used by public and private insurers to make quality-related decisions.13

Implications for the physician-learner from the literature reviewed in this chapter reinforce that multiple exposures to information in any educational activity are necessary to affect clinical outcomes and performance. However, to what degree medical education affects clinical outcomes and performance needs further exploration where studies can draw a potential significant correlation. Medical education often has been compared to the airline industry and its use of continuing safety education and teaching tools (eg, simulation) to reinforce and determine proficiency at performance. Should the future of CME be conducted in a similar fashion? With regard to the physician-teacher, CME activities of all types should be developed, studied, and chosen based ultimately on findings that indicate the greatest impact on clinical outcomes.

As detailed in the Methods article,9a the guideline panel reviewed the evidence tables as well as the comprehensive review of the effectiveness of CME report generated by The Johns Hopkins Evidence-based Practice Center.3 The present chapter reviews the effects of CME on clinical outcomes measured at < 30 days after the CME intervention and at ≥ 30 days after the CME.

Considering the quantity, quality, and consistency of evidence for the short- and long-term effects of CME on clinical outcomes, the strength of evidence is low. Thirty-seven studies evaluated the impact of CME on clinical outcomes, with one study measuring short-term clinical outcomes suggesting an inconclusive effect and three studies not reporting the point at which clinical outcomes were measured. Of the 33 studies that measured long-term clinical outcome,1446 only 13 studies22,25,3133,37,38,4045 showed a beneficial effect of CME. Although these studies represent a minority of the included studies, the panel believed that the potential for a beneficial effect of CME on clinical outcomes outweighed the perceived risks.

When evaluating the impact of different types of CME media on long-term clinical outcomes, the use of multiple media was more effective than the use of a single medium in six of seven studies22,25,28,31,32,37,46 that made the comparison. Three of five studies28,31,32,37,45 that compared clinical outcomes with use of single vs multiple educational techniques in CME showed multiple techniques to be superior to a single technique. Four of the seven studies that evaluated the impact of a single CME exposure on clinical outcomes met the instructional objective. Insufficient data were available to determine whether multiple CME exposures produced better clinical outcomes than a single exposure to CME.

Recommendations

  1. General: We suggest that CME activities be used to improve clinical practice outcomes (Grade 2C).

  2. Instructional media: We suggest the use of multiple media compared to single-medium interventions to improve clinical outcomes (Grade 2C).

  3. Instructional techniques: We suggest the use of multiple instructional techniques compared to single instructional techniques to improve clinical outcomes (Grade 2C).

  4. Frequency of exposure: We suggest multiple exposures to content to meet instructional objectives intended to improved clinical outcomes (Grade 2C).

The attempt to link CME interventions to improved clinical outcomes is not a new concept. Many studies have applied multiple interventions, some including quality improvement (QI) efforts. One such effort concluded that the short-term results of the educational intervention could not be interpreted separate from that of the QI intervention.47 A recently published systematic review4 of the effectiveness of teaching quality improvement to physicians concluded that most published QI curricula apply sound adult learning principles and demonstrate improvement in learners' knowledge or confidence to perform QI and that additional studies are required to determine whether educational methods have meaningful clinical benefits.

In 2001, an overview of systematic reviews of interventions intended to change provider behavior7 suggested that passive approaches are generally ineffective and unlikely to result in behavior change. The aim of the overview was to identify, appraise, and summarize systematic reviews of professional educational or quality assurance interventions to improve quality of care. Outcomes included such diverse measures as preventive care, test ordering, access to care, continuity of care, and physician–nurse collaboration. Most interventions were observed to be effective under some circumstances; none were effective under all circumstances. Although generalizable conclusions were not drawn, multifaceted interventions targeting barriers to change were more likely to be effective than single interventions in changing behavior and outcomes.

Another systematic review8 published in 2004 covered clinical guideline dissemination and implementation strategies and included single and multifaceted interventions to examine provider behavior and patient outcome. Randomized controlled trials, controlled clinical trials, controlled before-and-after studies, and interrupted time series qualified for the systematic review. Studies using aggregate self-report data on provider behavior (eg, “Do you usually measure patients' BP during visits?”) were excluded. Studies that evaluated only medical professionals' satisfaction and knowledge also were excluded. Commonly evaluated single interventions included reminders, educational materials, and audit and feedback. Multifaceted interventions often included educational outreach in combination with other interventions, with the majority of qualified interventions observing modest-to-moderate improvements in care,8 although many of the included studies6,8 had significant methodological weaknesses.

The evidence-based practice center review, as well as the reviews outlined previously,5,9 are consistent in the suggestion that assessment of need is required to reduce inconsistencies across practitioners, settings, and behaviors; interactive learning and opportunities to practice skills can effect practice change; and multifaceted activities can effect change in practice and outcomes. They also support the finding that future research should focus on developing a better theoretical understanding of the health-care professional and organizational behavior change,6 and they suggest the exploration of research and practice relationships of CME to guideline implementation, quality improvement, and clinical outcomes.

Outcomes can be defined as the result of efforts to prevent, diagnose, and treat various health problems encountered by a population.48 They are seen by many as the only legitimate measure for indicating the degree to which the health-care delivery system is achieving its purpose.48 Clinical and functional health status, patient experience, and provider experience comprise outcomes for use in assessing the performance of a health system. Clinical status; measurements of the biological, physiologic, and symptom-based aspects of health, including BP, blood glucose, and cholesterol; lung function; and mortality are examples of outcomes of interest to physicians partly because they are seen as amenable to treatment. Functional status includes physical, mental, role, and social functioning. Functional status is important to patients or consumers because it represents how changes in clinical status affect their everyday lives.12 Consumer satisfaction assesses the extent to which experience in the health-care system is consistent with expectations and acceptable to those receiving care.48 The 1989 Medical Outcomes Study (MOS)49 presented a conceptual framework that enabled the consideration of system characteristics, provider characteristics, and patient characteristics for measuring the structure, process, and outcomes of care (Table 3, Fig 1). The MOS short form 36 and the short form 12, which have psychometric origins that may be traced to the MOS, are now considered valid and reliable measures of health status. Studying their use may prove helpful to a fuller understanding of clinical outcomes and how CME interventions might improve them.

Table Graphic Jump Location
Table 3 Major Study Questions of the MOS49
Limitations

The Johns Hopkins Evidence-based Practice Center report3 carries limitations for those interested either in replicating the systematic review or in deeper exploration of the results. For example, on page 46, the investigators report, “Five studies compared single to multiple CME interventions.28,31,32,37,45 In three of the five studies, the use of multiple simultaneous CME techniques was superior to the use of a single CME technique.” As with the present review, readers are challenged to learn which three studies observed multiple techniques as superior to single techniques: the data are not readily available. In addition, the actual educational techniques used in each study are not outlined in detail in the report. More detailed discussions of the methods and limitations of the report3 may be found elsewhere in these guidelines.9a,9b

The evidence, although weak, supports the notion that CME activities should be used to improve clinical outcomes. It is currently impossible, however, to determine the extent to which the health-care system, the interdisciplinary health-care team, or the individual physician is responsible for observed outcomes. Health outcomes are affected by an assortment of factors, probably not all of which can be modified by the healthcare delivery system. One study50 estimated that 40% of mortality can be attributed to behavioral factors, 20% to social circumstances and environmental exposures, 30% to genetics, and 10% to inadequacies in medical care. Because these factors may be distributed differently among populations enrolled in health plans or in persons seeking the attention of primary-care physicians, it is essential to control for these external effects during statistical analyses intended to understand the extent to which variation in the quality of care contributes to observed variations in outcomes.48

Those who provide and study education and patient care must articulate sound explanations and realistic expectations for the value of their work; yet, the science to explain outcomes is incomplete and the causal linkages remain unclear.51 For patients with chronic disease, an outcome observed today may be the result of an action taken or health maintenance intervention forgotten earlier in the course of illness, and the action may or may not have been taken by the physician or health-care system currently responsible for treating the patient.48,52 To the extent that individuals change providers frequently, discontinuity in service may further contribute to a suboptimal course of treatment.48 Who bears responsibility for these complex series of events remains a question open to debate. The contention is driven, in part, by the need to identify qualities essential to measure the causal linkages thought to exist among CME, physician behavior, and clinical outcomes. The controversy must be addressed more thoroughly, as measured clinical outcomes become more central to the evaluation of health care and continuing education services are expected to support its systematic improvement.

CME

continuing medical education

HRQL

health-related quality of life

MOS

Medical Outcomes Study

QI

quality improvement

Dr. Mazmanian has no conflicts of interest.

Dr. Davis has no conflicts of interest.

Dr. Galbraith has no conflicts of interest.

Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65:S63-S67. [PubMed] [CrossRef]
 
Institute of Medicine Crossing the quality chasm: a new health system for the 21st century. 2001; Washington, DC National Academies Press
 
Marinopoulos SS, Dorman T, Ratanawongsa N, et al. Effectiveness of continuing medical education. Evid Rep Technol Assess (Full Rep).2007:1-69
 
Boonyasai RT, Windish DM, Chakraborti C, et al. Effectiveness of teaching quality improvement to clinicians: a systematic review. JAMA. 2007;298:1023-1037. [PubMed]
 
Davis DA, Thomson MA, Oxman AD, et al. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA. 1995;274:700-705. [PubMed]
 
Grimshaw J, Eccles M, Tetroe J. Implementing clinical guidelines: current evidence and future implications. J Contin Educ Health Prof. 2004;24suppl:S31-S37. [PubMed]
 
Grimshaw JM, Shirran L, Thomas R, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001;39:II2-II45. [PubMed]
 
Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004;8:iii-iv:1-72. [PubMed]
 
Mazmanian PE, Davis DA. Continuing medical education and the physician as a learner: guide to the evidence. JAMA. 2002;288:1057-1060. [PubMed]
 
Marinopoulos SS, Baumann MH. Methods and definitions of terms: effectiveness of continuing medical education: American College of Chest Physicians evidence-based educational guidelines. Chest. 2009;135suppl:17S-28S. [PubMed]
 
Bordage G, Carlin B, Mazmanian PE. Continuing medical education effect on physician knowledge: effectiveness of continuing medical education: American College of Chest Physicians evidence-based educational guidelines. Chest. 2009;135suppl:29S-36S. [PubMed]
 
Moore D.Davis D, Barnes B, Fox R. A framework for outcomes evaluation in the continuing professional development of physicians. The continuing professional development of physicians. 2007; Chicago, IL American Medical Association Press:249-274
 
Accreditation Council for Continuing Medical Education Decision-making criteria relevant to the essential areas and elements: 2006 udpate.Accessed January 12, 2009 Available at:http://www.accme.org/dir_docs/doc_upload/b03aa5cc-b017–4395-a41f–8d5d89ac31ca_uploaddocument.pdf.
 
Ganz PA, Litwin MS, Hays RD, et al;Andersen RM, Rice TH, Kominski GF. Measuring outcomes in health-related quality of life. Changing the US health care system. 2007; San Francisco, CA Jossey-Bass:185-211
 
Institute of MedicineOlsen L, Aisner D, McGinnis JM. The learning healthcare system: workshop summary. 2007; Washington, DC National Academies Press:289-313
 
Brown JB, Boles M, Mullooly JP, et al. Effect of clinician communication skills training on patient satisfaction: a randomized, controlled trial. Ann Intern Med. 1999;131:822-829. [PubMed]
 
Brown R, Bratton SL, Cabana MD, et al. Physician asthma education program improves outcomes for children of low-income families. Chest. 2004;126:369-374. [PubMed]
 
Casebeer LL, Klapow JC, Centor RM, et al. An intervention to increase physicians' use of adherence-enhancing strategies in managing hypercholesterolemic patients. Acad Med. 1999;74:1334-1339. [PubMed]
 
Clark NM, Gong M, Schork MA, et al. Impact of education for physicians on patient outcomes. Pediatrics. 1998;101:831-836. [PubMed]
 
Clark NM, Gong M, Schork MA, et al. Long-term effects of asthma education for physicians on patient satisfaction and use of health services. Eur Respir J. 2000;16:15-21. [PubMed]
 
Cummings SR, Coates TJ, Richard RJ, et al. Training physicians in counseling about smoking cessation: a randomized trial of the “Quit for Life” program. Ann Intern Med. 1989;110:640-647. [PubMed]
 
Cummings SR, Richard RJ, Duncan CL, et al. Training physicians about smoking cessation: a controlled trial in private practices. J Gen Intern Med. 1989;4:482-489. [PubMed]
 
Cummings SR, Richard RJ, Duncan CL, et al. Training physicians about smoking cessation: a controlled trial in private practice. J Gen Intern Med. 1989;4:482-489. [PubMed]
 
Derebery VJ, Giang GM, Saracino G, et al. Evaluation of the impact of a low back pain educational intervention on physicians' practice patterns and patients' outcomes. J Occup Environ Med. 2002;44:977-984. [PubMed]
 
Elliott TE, Murray DM, Oken MM, et al. Improving cancer pain management in communities: main results from a randomized controlled trial. J Pain Symptom Manage. 1997;13:191-203. [PubMed]
 
Evans CE. Does a mailed continuing education program improve physician performance? Results of a randomized trial in antihypertensive care. JAMA. 1986;255:501-504. [PubMed]
 
Gonzales R, Steiner JF, Lum A, et al. Decreasing antibiotic use in ambulatory practice: impact of a multidimensional intervention on the treatment of uncomplicated acute bronchitis in adults. JAMA. 1999;281:1512-1519. [PubMed]
 
Gullion DS, et al. Management of hypertension in private practice: a randomized controlled trial in continuing medical education. J Contin Educ Health Prof. 1988;8:239-255
 
Harris SB, Leiter LA, Webster-Bogaert S, et al. Teleconferenced educational detailing: diabetes education for primary care physicians. J Contin Educ Health Prof. 2005;25:87-97. [PubMed]
 
Kim CS, Kristopaitis RJ, Stone E, et al. Physician education and report cards: do they make the grade? Results from a randomized controlled trial. Am J Med. 1999;107:556-560. [PubMed]
 
Kutcher SP, Lauria-Horner BA, MacLaren CM, et al. Evaluating the impact of an educational program on practice patterns of Canadian family physicians interested in depression treatment. Prim Care Companion J Clin Psychiatry. 2002;4:224-231. [PubMed]
 
Lin EH, Katon WJ, Simon GE, et al. Achieving guidelines for the treatment of depression in primary care: is physician education enough? Med Care. 1997;35:831-842. [PubMed]
 
Lindsay-McIntyre E, Wilson D, Best JA, et al. The impact of a continuing education package for smoking cessation on physicians' clinical behavior and patient smoking. Proc Annu Conf Res Med Educ. 1987;26:14-19
 
Maiman LA, Becker MH, Liptak GS, et al. Improving pediatricians' compliance-enhancing practices: a randomized trial. Am J Dis Child. 1988;142:773-779. [PubMed]
 
McMahon SM, McKenna P, Hodgins JL. An application of continuing medical education to decrease excessive lengths of stay. J Med Educ. 1988;63:364-371. [PubMed]
 
Mehler PS, Krantz MJ, Lundgren RA, et al. Bridging the quality gap in diabetic hyperlipidemia: a practice-based intervention. Am J Med. 2005;118:1414. [PubMed]
 
Messina CR, Lane DS, Grimson R. Effectiveness of women's telephone counseling and physician education to improve mammography screening among women who underuse mammography. Ann Behav Med. 2002;24:279-289. [PubMed]
 
Norris SL, Grothaus LC, Buchner DM, et al. Effectiveness of physician-based assessment and counseling for exercise in a staff model HMO. Prev Med. 2000;30:513-523. [PubMed]
 
Pinto BM, Goldstein MG, DePue JD, et al. Acceptability and feasibility of physician-based activity counseling: the PAL project. Am J Prev Med. 1998;15:95-102. [PubMed]
 
Ray WA, Stein CM, Byrd V, et al. Educational program for physicians to reduce use of non-steroidal anti-inflammatory drugs among community-dwelling elderly persons: a randomized controlled trial. Med Care. 2001;39:425-435. [PubMed]
 
Rodney WM, Albers G. Flexible sigmoidoscopy: primary care outcomes after two types of continuing medical education. Am J Gastroenterol. 1986;81:133-137. [PubMed]
 
Rost K, Nutting P, Smith J, et al. Improving depression outcomes in community primary care practice: a randomized trial of the quEST intervention: quality enhancement by strategic teaming. J Gen Intern Med. 2001;16:143-149. [PubMed]
 
Roter DL, Hall JA, Kern DE, et al. Improving physicians' interviewing skills and reducing patients' emotional distress: a randomized clinical trial. Arch Intern Med. 1995;155:1877-1884. [PubMed]
 
Stein CM, Griffin MR, Taylor JA, et al. Educational program for nursing home physicians and staff to reduce use of non-steroidal anti-inflammatory drugs among nursing home residents: a randomized controlled trial. Med Care. 2001;39:436-445. [PubMed]
 
Stewart M, Marshall JN, Ostbye T, et al. Effectiveness of case-based on-line learning of evidence-based practice guidelines. Fam Med. 2005;37:131-138. [PubMed]
 
Wilson DM, Taylor DW, Gilbert JR, et al. A randomized trial of a family physician intervention for smoking cessation. JAMA. 1988;260:1570-1574. [PubMed]
 
Worrall G, Angel J, Chaulk P, et al. Effectiveness of an educational strategy to improve family physicians' detection and management of depression: a randomized controlled trial. Can Med Assoc J. 1999;161:37-40
 
Kottke TE, Brekke ML, Solberg LI, et al. A randomized trial to increase smoking intervention by physicians: doctors helping smokers, round I. JAMA. 1989;261:2101-2106. [PubMed]
 
Gerrity MS, Cole SA, Dietrich AJ, et al. Improving the recognition and management of depression: is there a role for physician education? J Fam Pract. 1999;48:949-957. [PubMed]
 
McGlynn EA.Andersen RM, Rice TH, Kominski GF. Evaluating the quality of care. Changing the US health care system. 2007; San Francisco, CA Jossey-Bass:213-244
 
Tarlov AR, Ware JE Jr, Greenfield S, et al. The Medical Outcomes Study: an application of methods for monitoring the results of medical care. JAMA. 1989;262:925-930. [PubMed]
 
McGinnis JM, Williams-Russo P, Knickman JR. The case for more active policy attention to health promotion. Health Aff (Millwood). 2002;21:78-93. [PubMed]
 
Mazmanian P.Davis D, Barnes B, Fox R. Advancing the body of knowledge: evidence and study design for quality improvment. The continuing professional development of physicians: from research to practice. 2003; Chicago, IL American Medical Association Press:317-339
 
Frame PS.Woolf SH, Jonas S, Kaplan-Liss E. Developing a health maintenance schedule. Health promotion and disease prevention in clinical practice. 2007;2nd ed. Philadelphia, PA Lippincott Williams & Wilkins:517-530
 

Figures

Figure Jump LinkFigure 1 The conceptual framework of the MOS.Grahic Jump Location

Tables

Table Graphic Jump Location
Table 1 Summary of CME Activity Characteristics*

* Values are given as No. (%). From Marinopoulos et al3 (Table 4, page 26).

Table Graphic Jump Location
Table 2 Levels of an Outcomes-Based CME Evaluation Model*

* From Moore10 (Table 13-1, page 251).

Table Graphic Jump Location
Table 3 Major Study Questions of the MOS49

References

Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65:S63-S67. [PubMed] [CrossRef]
 
Institute of Medicine Crossing the quality chasm: a new health system for the 21st century. 2001; Washington, DC National Academies Press
 
Marinopoulos SS, Dorman T, Ratanawongsa N, et al. Effectiveness of continuing medical education. Evid Rep Technol Assess (Full Rep).2007:1-69
 
Boonyasai RT, Windish DM, Chakraborti C, et al. Effectiveness of teaching quality improvement to clinicians: a systematic review. JAMA. 2007;298:1023-1037. [PubMed]
 
Davis DA, Thomson MA, Oxman AD, et al. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA. 1995;274:700-705. [PubMed]
 
Grimshaw J, Eccles M, Tetroe J. Implementing clinical guidelines: current evidence and future implications. J Contin Educ Health Prof. 2004;24suppl:S31-S37. [PubMed]
 
Grimshaw JM, Shirran L, Thomas R, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001;39:II2-II45. [PubMed]
 
Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004;8:iii-iv:1-72. [PubMed]
 
Mazmanian PE, Davis DA. Continuing medical education and the physician as a learner: guide to the evidence. JAMA. 2002;288:1057-1060. [PubMed]
 
Marinopoulos SS, Baumann MH. Methods and definitions of terms: effectiveness of continuing medical education: American College of Chest Physicians evidence-based educational guidelines. Chest. 2009;135suppl:17S-28S. [PubMed]
 
Bordage G, Carlin B, Mazmanian PE. Continuing medical education effect on physician knowledge: effectiveness of continuing medical education: American College of Chest Physicians evidence-based educational guidelines. Chest. 2009;135suppl:29S-36S. [PubMed]
 
Moore D.Davis D, Barnes B, Fox R. A framework for outcomes evaluation in the continuing professional development of physicians. The continuing professional development of physicians. 2007; Chicago, IL American Medical Association Press:249-274
 
Accreditation Council for Continuing Medical Education Decision-making criteria relevant to the essential areas and elements: 2006 udpate.Accessed January 12, 2009 Available at:http://www.accme.org/dir_docs/doc_upload/b03aa5cc-b017–4395-a41f–8d5d89ac31ca_uploaddocument.pdf.
 
Ganz PA, Litwin MS, Hays RD, et al;Andersen RM, Rice TH, Kominski GF. Measuring outcomes in health-related quality of life. Changing the US health care system. 2007; San Francisco, CA Jossey-Bass:185-211
 
Institute of MedicineOlsen L, Aisner D, McGinnis JM. The learning healthcare system: workshop summary. 2007; Washington, DC National Academies Press:289-313
 
Brown JB, Boles M, Mullooly JP, et al. Effect of clinician communication skills training on patient satisfaction: a randomized, controlled trial. Ann Intern Med. 1999;131:822-829. [PubMed]
 
Brown R, Bratton SL, Cabana MD, et al. Physician asthma education program improves outcomes for children of low-income families. Chest. 2004;126:369-374. [PubMed]
 
Casebeer LL, Klapow JC, Centor RM, et al. An intervention to increase physicians' use of adherence-enhancing strategies in managing hypercholesterolemic patients. Acad Med. 1999;74:1334-1339. [PubMed]
 
Clark NM, Gong M, Schork MA, et al. Impact of education for physicians on patient outcomes. Pediatrics. 1998;101:831-836. [PubMed]
 
Clark NM, Gong M, Schork MA, et al. Long-term effects of asthma education for physicians on patient satisfaction and use of health services. Eur Respir J. 2000;16:15-21. [PubMed]
 
Cummings SR, Coates TJ, Richard RJ, et al. Training physicians in counseling about smoking cessation: a randomized trial of the “Quit for Life” program. Ann Intern Med. 1989;110:640-647. [PubMed]
 
Cummings SR, Richard RJ, Duncan CL, et al. Training physicians about smoking cessation: a controlled trial in private practices. J Gen Intern Med. 1989;4:482-489. [PubMed]
 
Cummings SR, Richard RJ, Duncan CL, et al. Training physicians about smoking cessation: a controlled trial in private practice. J Gen Intern Med. 1989;4:482-489. [PubMed]
 
Derebery VJ, Giang GM, Saracino G, et al. Evaluation of the impact of a low back pain educational intervention on physicians' practice patterns and patients' outcomes. J Occup Environ Med. 2002;44:977-984. [PubMed]
 
Elliott TE, Murray DM, Oken MM, et al. Improving cancer pain management in communities: main results from a randomized controlled trial. J Pain Symptom Manage. 1997;13:191-203. [PubMed]
 
Evans CE. Does a mailed continuing education program improve physician performance? Results of a randomized trial in antihypertensive care. JAMA. 1986;255:501-504. [PubMed]
 
Gonzales R, Steiner JF, Lum A, et al. Decreasing antibiotic use in ambulatory practice: impact of a multidimensional intervention on the treatment of uncomplicated acute bronchitis in adults. JAMA. 1999;281:1512-1519. [PubMed]
 
Gullion DS, et al. Management of hypertension in private practice: a randomized controlled trial in continuing medical education. J Contin Educ Health Prof. 1988;8:239-255
 
Harris SB, Leiter LA, Webster-Bogaert S, et al. Teleconferenced educational detailing: diabetes education for primary care physicians. J Contin Educ Health Prof. 2005;25:87-97. [PubMed]
 
Kim CS, Kristopaitis RJ, Stone E, et al. Physician education and report cards: do they make the grade? Results from a randomized controlled trial. Am J Med. 1999;107:556-560. [PubMed]
 
Kutcher SP, Lauria-Horner BA, MacLaren CM, et al. Evaluating the impact of an educational program on practice patterns of Canadian family physicians interested in depression treatment. Prim Care Companion J Clin Psychiatry. 2002;4:224-231. [PubMed]
 
Lin EH, Katon WJ, Simon GE, et al. Achieving guidelines for the treatment of depression in primary care: is physician education enough? Med Care. 1997;35:831-842. [PubMed]
 
Lindsay-McIntyre E, Wilson D, Best JA, et al. The impact of a continuing education package for smoking cessation on physicians' clinical behavior and patient smoking. Proc Annu Conf Res Med Educ. 1987;26:14-19
 
Maiman LA, Becker MH, Liptak GS, et al. Improving pediatricians' compliance-enhancing practices: a randomized trial. Am J Dis Child. 1988;142:773-779. [PubMed]
 
McMahon SM, McKenna P, Hodgins JL. An application of continuing medical education to decrease excessive lengths of stay. J Med Educ. 1988;63:364-371. [PubMed]
 
Mehler PS, Krantz MJ, Lundgren RA, et al. Bridging the quality gap in diabetic hyperlipidemia: a practice-based intervention. Am J Med. 2005;118:1414. [PubMed]
 
Messina CR, Lane DS, Grimson R. Effectiveness of women's telephone counseling and physician education to improve mammography screening among women who underuse mammography. Ann Behav Med. 2002;24:279-289. [PubMed]
 
Norris SL, Grothaus LC, Buchner DM, et al. Effectiveness of physician-based assessment and counseling for exercise in a staff model HMO. Prev Med. 2000;30:513-523. [PubMed]
 
Pinto BM, Goldstein MG, DePue JD, et al. Acceptability and feasibility of physician-based activity counseling: the PAL project. Am J Prev Med. 1998;15:95-102. [PubMed]
 
Ray WA, Stein CM, Byrd V, et al. Educational program for physicians to reduce use of non-steroidal anti-inflammatory drugs among community-dwelling elderly persons: a randomized controlled trial. Med Care. 2001;39:425-435. [PubMed]
 
Rodney WM, Albers G. Flexible sigmoidoscopy: primary care outcomes after two types of continuing medical education. Am J Gastroenterol. 1986;81:133-137. [PubMed]
 
Rost K, Nutting P, Smith J, et al. Improving depression outcomes in community primary care practice: a randomized trial of the quEST intervention: quality enhancement by strategic teaming. J Gen Intern Med. 2001;16:143-149. [PubMed]
 
Roter DL, Hall JA, Kern DE, et al. Improving physicians' interviewing skills and reducing patients' emotional distress: a randomized clinical trial. Arch Intern Med. 1995;155:1877-1884. [PubMed]
 
Stein CM, Griffin MR, Taylor JA, et al. Educational program for nursing home physicians and staff to reduce use of non-steroidal anti-inflammatory drugs among nursing home residents: a randomized controlled trial. Med Care. 2001;39:436-445. [PubMed]
 
Stewart M, Marshall JN, Ostbye T, et al. Effectiveness of case-based on-line learning of evidence-based practice guidelines. Fam Med. 2005;37:131-138. [PubMed]
 
Wilson DM, Taylor DW, Gilbert JR, et al. A randomized trial of a family physician intervention for smoking cessation. JAMA. 1988;260:1570-1574. [PubMed]
 
Worrall G, Angel J, Chaulk P, et al. Effectiveness of an educational strategy to improve family physicians' detection and management of depression: a randomized controlled trial. Can Med Assoc J. 1999;161:37-40
 
Kottke TE, Brekke ML, Solberg LI, et al. A randomized trial to increase smoking intervention by physicians: doctors helping smokers, round I. JAMA. 1989;261:2101-2106. [PubMed]
 
Gerrity MS, Cole SA, Dietrich AJ, et al. Improving the recognition and management of depression: is there a role for physician education? J Fam Pract. 1999;48:949-957. [PubMed]
 
McGlynn EA.Andersen RM, Rice TH, Kominski GF. Evaluating the quality of care. Changing the US health care system. 2007; San Francisco, CA Jossey-Bass:213-244
 
Tarlov AR, Ware JE Jr, Greenfield S, et al. The Medical Outcomes Study: an application of methods for monitoring the results of medical care. JAMA. 1989;262:925-930. [PubMed]
 
McGinnis JM, Williams-Russo P, Knickman JR. The case for more active policy attention to health promotion. Health Aff (Millwood). 2002;21:78-93. [PubMed]
 
Mazmanian P.Davis D, Barnes B, Fox R. Advancing the body of knowledge: evidence and study design for quality improvment. The continuing professional development of physicians: from research to practice. 2003; Chicago, IL American Medical Association Press:317-339
 
Frame PS.Woolf SH, Jonas S, Kaplan-Liss E. Developing a health maintenance schedule. Health promotion and disease prevention in clinical practice. 2007;2nd ed. Philadelphia, PA Lippincott Williams & Wilkins:517-530
 
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Find Similar Articles
CHEST Journal Articles
PubMed Articles
  • CHEST Journal
    Print ISSN: 0012-3692
    Online ISSN: 1931-3543