ACCP Evidence-Based Guideline Development^*: A Successful and Transparent Approach Addressing Conflict of Interest, Funding, and Patient-Centered Recommendations FREE TO VIEW

Evidence-based clinical practice guidelines (EBGs) can provide an invaluable distillation of knowledge regarding best practices based on the available evidence. A host of EBGs dealing with many clinical conditions exist today,¹² but the variability in quality may be high.³ Incorporation of high-quality EBGs into daily clinical care by health-care providers has been limited; nevertheless, when successfully implemented, patient and health-care utilization outcomes improve.⁴ Buoyed by these successes, clinical practice EBGs are being utilized in the development of clinical order sets, performance measures, and pay for performance.

EBGs providing accurate and useful guidance to best clinical practices require a rigorous development process. The American College of Chest Physicians (ACCP) has developed and refined such a process that has resulted in the publication of nine guidelines over the last 7 years. Refinements have focused on making the process transparent, rigorous, and timely; effectively managing conflicts of interest; and employing a quantitative and rigorous grading of the strength of recommendations and of the quality of evidence that incorporates sensitivity to health-care resource utilization and patient preferences.⁵⁶ A complete description of the ACCP evidence-based guideline development process is available on the ACCP Web site⁷ and in the ACCP Manual for Guideline Development. In this article, we review our process for guideline development to educate ACCP members and to provide a framework that others may find useful in developing EBGs. The process is dynamic and continues to undergo improvements reflecting the need to adapt to an ever-changing and complex health-care environment, and to provide the best available educational products for the care of our patients.

EBGs are defined as a systematically developed set of recommendations, algorithms, and other information to assist health-care decision making in specific clinical circumstances. EBGs represent the synthesis of evidence derived from a formal, thorough, and systematic review of the literature that results in the compilation of a set of specific literature-based recommendations addressing specifically identified clinical questions. An EBG must be distinguished from a consensus statement, which is defined as a collective opinion of a convened expert panel. The opinions expressed in the consensus statement are also derived from existing literature but usually only when the amount and quality of the available data are insufficient for a formal evidence-based review. “Suggestions” are developed by a standardized process in a consensus statement as opposed to “recommendations” presented in an EBG following a rigid literature review.⁷

The ACCP Health and Science Policy Committee (HSP) provides oversight for the development of ACCP-generated EBGs. It is charged with the responsibility of choosing the topic and maintaining the integrity of all EBGs as well as the process used to develop them. The HSP ensures that EBGs are scientifically sound and are based on the available evidence as determined by rigorous reviews of the literature. Further, the HSP continuously evaluates and improves the rigorous basis for this development process to maintain the highest quality and to manage conflicts of interest.

The HSP supervises a regimented and transparent approach to EBG development according to the representative time line shown in Table 1 . This process dictates the methods used for analyses and synthesis of data, guides the creation and utilization of evidence tables for recommendation generation, and ensures the appropriate use of the ACCP-generated grading system for grading recommendations based on the quality of the evidence and the balance of benefits and harms associated with each recommendation.⁵

New EBG topics are generated through an application process by individual ACCP members or by a committee of the ACCP. These suggestions are brought to the HSP for assessment. The application requires an assessment of the importance of the topic, including the variability in practice patterns, the level of controversy regarding the topic, the availability of interventions that have high societal or economic cost, the breadth of the constituency impacted, the importance to public health, and the availability and quality of the published evidence, including that of existing relevant guidelines. EBG topics proposed by the HSP as well as related development expenses must be approved by the ACCP Board of Regents. Funds are approved for the entirety of the project (ie, a multiyear commitment) contingent on project progress reports to the Board of Regents.

Early in the guideline development process, variable degrees of collaboration are sought with relevant organizations and medical specialty societies that bring important perspectives such as complementary clinical expertise or patient advocacy. A Memorandum of Agreement defines organizational responsibilities and expenses, as well as the conditions for publication, authorship, the management of conflicts of interest, endorsement, profits, and termination. This memorandum is customized to meet the needs of the guideline collaboration opportunity.

The “Conflict of Interest Policy for ACCP Guideline Development”⁷ outlines a process ensuring that disclosed conflicts of interest are properly evaluated and resolved at several key points during the development of the EBG. This policy statement includes an explicit and detailed step-by-step procedure to collect and evaluate the disclosed conflicts respective to the guideline topic, and make recommendations to resolve or manage the conflict. The conflict-of-interest assessment flowchart (Fig 1 ) shows how disclosures are reviewed along with the resulting actions. EBG panel members are requested to disclose in writing conflicts at several times during the course of EBG development, as follows: at the time panelists are nominated for participation on the guideline development panel; at every face-to-face meeting; and prior to publication in the journal CHEST.

ACCP reviewers of EBG guidelines (eg, the HSP, Board of Regents, and NetWork members) must adhere to current ACCP conflict-of-interest policies and disclose all conflicts (ie, personal as well as financial). NetWorks are the ACCP interdisciplinary, expertise-focused interest groups that provide college members the opportunity for personal and professional alliances with the ACCP.

If an ACCP EBG receives financial support from industry (including the pharmaceutical industry) or foundations, the financial support must be unrestricted, and the funds are invested into a rigorous methodological process that is used by the EBG clinical panel of experts (ie, the writing panel) to formulate graded recommendations, as well as the operational costs of developing the guidelines, including the evidence review, meetings, travel, production, editing, publicizing, printing, and distribution. External support of guideline development is independent of the guideline process. Contributing industries or other entities may not have representation on writing panels, evidence-based practice center review groups, or EBG writing panels. If an EBG has industry support, the ACCP seeks to obtain multiple sponsors. Nearly one third of the EBGs in the last 6 years have received no external support, with funding allocated by the Board of Regents. However, most of the other guidelines had the support of three to five sponsors each. All expenditures of donated funds are approved by the HSP. A donor may not target the use of funds other than to direct that the funds be used to support an associated “clinical resource tool” (see the “Dissemination and Marketing” section) and/or to support publication costs. This policy minimizes bias or the perception of bias during the development of an EBG or the content of any derivative educational programs or products and protects all those participating in EBG development. For these mutual protections, the HSP enforces the following policies:

Methodologic reviews must be conducted by an evidence-based practice center that is independent of the EBG clinical panel of experts (ie, the writing panel). This separation of grading and the EBG writing panel complies with the ACCP conflict-of-interest policies. Members of an evidence-based practice center conducting an evidence review and contracted directly by the ACCP must also comply with all aspects of the ACCP conflict-of-interest policy.

Chairs discuss the initial scope of the guideline and suggest experts in each subtopic. The initial literature review helps to identify content experts who have extensive publication experience in the clinical area. The selection process identifies panel members. In addition to strong clinical and methodological expertise, selection criteria include reliability, the ability to work collaboratively, anticipated productivity, and gender, minority, and geographic diversity.

All EBG writing panel nominees must submit a curriculum vitae or biosketch and a completed standardized conflict-of-interest form that is tailored to the EBG topic. All materials are reviewed by a subcommittee of the HSP to determine whether the nominee (1) meets the criteria for authorship, and (2) has no significant real or perceived conflicts of interest. Panel involvement by research assistants, residents, fellows, and junior faculty either as members or substantive contributors is discouraged. To be eligible for chapter authorship of an ACCP EBG, panelists must agree to the following:

The ACCP does not provide honoraria to members of an EBG writing panel. However, travel and select miscellaneous expenses for face-to-face meetings related to the guideline development are reimbursed as described in the “Policy Statement on Honoraria and Travel Expenses.”⁷

A project time line (Table 1) is drafted once the decision is made to develop an EBG and the project is approved by the Board of Regents. The ACCP project manager and EBG Executive Committee (composed of the chair, the methodologist, and a liaison from the HSP) are responsible for keeping the entire panel within the project deadlines. The HSP is regularly updated.

The EBG Executive Committee and the ACCP staff project manager convene regular conference calls to review administrative and content issues, and to assess and maintain progress. The HSP liaison ensures that the HSP EBG development process is followed.

Typically two face-to-face meetings of the EBG writing panel are scheduled. The first is held with the panelists shortly after their participation is approved. At the meeting, the panel finalizes the project scope, addresses any unfilled positions, develops the research questions to be answered by the evidence review, and receives education on the ACCP grading system⁵ and HSP EBG process, including matters of writing format and style. The second meeting occurs after a revised draft of the EBG has been completed. This meeting is used to provide wide input regarding recommendations prior to submission to the HSP and NetWorks for critique. Disagreements in recommendations are resolved at this meeting.

Once a proposed EBG topic is selected for development, the HSP determines how the evidence review will be conducted. The ACCP typically contracts with an Agency for Healthcare Research and Quality (AHRQ)-approved evidence-based practice center to review the literature and prepare evidence tables for the writing committee. In some cases, a request is submitted to the AHRQ and, if approved, the AHRQ recruits an evidence-based practice center to conduct the evidence review for a portion or all of the EBG. In this case, the EBG research questions are refined by an AHRQ-appointed technical expert panel, often including a member of the ACCP EBG Executive Committee. The research (evidence) questions are formulated to specifically define the appropriate patient population, intervention/exposure/comparison, outcomes of interest, and inclusion/exclusion criteria. The review is focused on all relevant evidence, ranking the quality of the evidence according to the ACCP guidelines.⁵ The resulting evidence and evidence summary tables are then utilized by the ACCP EBG writing panel to develop recommendations.

When the ACCP directly contracts for an evidence review with an AHRQ-approved evidence-based practice center, a request for proposals is sent to several evidence-based practice centers for a bid-based selection process. If the EBG is relatively narrow in scope, as defined by the extent of evidence to be reviewed, an ACCP staff methodologist conducts the evidence review and develops the evidence tables through a very similar process. An ACCP staff methodologist may also supplement an evidence review and evidence table development when an evidence-based practice center reviews only part of the relevant evidence.

The systematic review of the evidence commences shortly after the research questions are finalized. Systematic reviews and metaanalyses are identified early, and source articles are obtained to verify whether they meet the inclusion criteria. Data abstractions, development of the evidence tables, and grading of the evidence take several months for most guidelines, regardless of whether an AHRQ-contracted evidence-based practice center, ACCP-contracted evidence-based practice center, or ACCP staff methodologist reviews the evidence. The content experts (ie, writing panelists) review the list of references to identify any that might have been omitted. These content experts, usually the chapter editors, are requested to carefully review the evidence tables before the content is synthesized for the summary tables.

A detailed description of the ACCP EBG writing standards is available on the HSP Web site.⁷ Continuity and consistency of style from guideline to guideline is an explicit goal of this process. It is the responsibility of the EBG Executive Committee and ACCP project manager to ensure that the EBG writing panel adheres to these standards.

Chapter chairs work closely with their writing panel groups (coauthors), ensuring that all draft recommendations are cognitively aligned with the data provided in the evidence tables. Uniformity of style and consistency of the content of the recommendations across and within chapters, whether it is a single-chapter or multichapter EBG, are key objectives. The EBG Executive Committee reviews the content and facilitates the resolution of any inconsistencies. The general format for each recommendation is found in Table 2 .

A robust ACCP grading system was developed by a task force composed of individuals with significant experience in guideline development and grading recommendations. This task force developed a system⁵ that can be “translated” to prior ACCP grading systems and to several other widely used grading systems, facilitating comparisons among guideline recommendations.⁹ The grading system considers both the quality of the evidence and the balance of benefits to risk and burdens. It is designed to be robust, accurate, and maximally informative, as well as user-friendly and understandable to the nonexpert user.

An important but often overlooked issue has been how resource constraints impact the applicability and implementation of guideline recommendations.⁶ Because of this, the ACCP is incorporating resource allocation into targeted recommendations for the upcoming guideline on antithrombotic and thrombolytic therapy (Antithrombotic and Thrombolytic Therapy: ACCP Evidence-Based Clinical Practice Guidelines [Eighth Edition]; unpublished data) that is now in the final stages of development. Expert panelists have incorporated economic analyses into a limited number of recommendations in which a wide range of resource constraints exists across various geographic areas and population groups. Recommendations incorporating resource allocation or cost-benefit analyses are formatted as outlined above; however, they may be downgraded and include an additional explanatory statement.

The previously mentioned eighth edition of the antithrombotic and thrombolytic therapy guidelines now address patient values and preferences by several experts in the field. These experts assessed the potential impact of patient-important outcomes and reworded recommendations in some chapters to incorporate underlying patient values and preferences. For example, in the chapter on thromboembolism and thrombophilia in pregnancy, the mother’s concerns for the health of her fetus was considered to have an impact on the recommendation to use potentially teratogenic anticoagulants. The incorporation of patient values should be considered early in EBG development. Our goal is to incorporate resource assessment and patient values and preference considerations in all relevant recommendations.

Once the evidence tables are finalized, and the initial chapter drafts are completed, reviewed, and revised by the writing panel, the final EBG writing panel face-to-face meeting is used to solidify the acceptance of the final product by the panel. During the meeting, any significant unresolved issues and controversies are identified, discussed by the full panel, and resolved by vote if needed. Areas open for discussion and debate include the wording of recommendations, the balance of risks to benefits, and the interpretation of the strength of evidence where an evidence-based practice center has not made a determination. If ≥ 80% of the writing panel is in agreement with a recommendation, the recommendation is approved. At the discretion of the chair, (1) discussions may continue until there is a supermajority (ie, > 80%) agreement, (2) the recommendation can be removed for lack of agreement, or (3) the recommendation can remain but a minority opinion included within the discussion preceding the recommendation.

Based on the proceedings of the final conference, revisions are made to the guideline that is then forwarded to several groups within the ACCP for review and revision. (Fig 2 .) Appropriate ACCP NetWorks are charged with content review. The HSP reviews process, consistency, whether the recommendations and grading are appropriate, and content. After the writing group adequately addresses the critiques provided by HSP and NetWork reviewers, the guideline manuscript is submitted to the ACCP Board of Regents for final approval. A structured review form, based in part on the Appraisal of Guidelines Research and Evaluation instrument,³ (Table 3 ), including a grid for comments, is used by the HSP, NetWork, and Board of Regents reviewers to ensure a complete review.

Once approved by the Board of Regents, the manuscript is submitted to CHEST for the consideration of publication and external independent review, which is performed according to the standard editorial policies of CHEST. The EBG review process adopted by the ACCP balances thoroughness with expediency while providing the highest possible quality and transparency. The rationale for expediency stems from the fact that the timeliness of all recommendations begins to decline as soon as the evidence review ends.

A “Clinical Resource Tool” is created for most ACCP EBGs. This product is a combination print and CD-ROM implementation tool kit that is based on the clinical practice guideline. Standard components include the Quick Reference Guide for Clinicians, patient education materials, and slides for presentations to lay and medical audiences. Additional materials, such as physician order sets, can also be added. The Quick Reference Guide for Clinicians includes all clinical algorithms and the key recommendations, along with their grading in print and electronic format or downloadable to personal digital assistants (ie, PDAs).

EBGs are reviewed annually by the HSP (and by appropriate NetWorks and in consultation with the original EBG Executive Committee) to determine whether the recommendations remain current or whether interim studies have provided sufficient information to warrant revision. One of the following four possible EBG status categories is applied to each guideline every year beginning 1 year after the initial publication:

The ACCP Quality Improvement Committee works collaboratively with the HSP through the EBG HSP liaison to the EBG writing panels to identify recommendations that could be developed into performance measures for improving patient care. Based on the criterion of the National Quality Forum for a reasonable performance measure,¹⁰ a performance measure should be scientifically acceptable, important, feasible, and usable. Additionally, the measure should be practical and relevant for ACCP members and their patients.¹¹ Concomitantly, recommendations that should not be developed into performance measures are also identified. These latter recommendations usually have a lower quality of evidence to support them, although there could also be components of the intervention that are unfeasible or unusable. Performance measures may be incorporated into pay-for-performance strategies (so-called value-based performance strategies) that are currently under development across many sectors of the health care field in the United States.¹² Whether pay-for-performance strategies are successful and equitable for patients and health-care providers will inextricably depend on the quality of the performance measures implemented.

The HSP and the Quality Improvement Committee work collaboratively to develop implementation strategies to increase the adoption of EBG recommendations by front-line health-care providers. This is a particularly challenging issue for all guideline developers. Several nascent strategies are being developed and tested on a small scale by the ACCP HSP and Quality Improvement Committee. Integral to these strategies is ongoing coordination with the ACCP Education Committee for ensuring the quality of educational opportunities related to any EBG. The Education Committee advises on and reviews the development of all educational courses and course materials, and participates in the selection of faculty for courses offered by the ACCP. Successful strategies will be posted on the HSP⁷ and Quality Improvement Committee¹¹ Web sites as they evolve.

EBG development is a collaborative and complex endeavor. However, when parsed into specific domains including topic selection, writing panel selection, conflict-of-interest declaration and resolution, an explicit time line, evidence grading and recommendation development, comprehensive guideline review, and creating and maintaining appropriate and robust firewalls, the process is transparent and manageable. We present a systematic strategy that has been successful for the ACCP. However, this approach continues to evolve with each EBG that is developed. At the core of each evolutionary step is the premise of producing the highest quality EBG possible, using explicit recommendation development based on rigorous evidence review while maintaining transparency and internal and external quality checks throughout the process. We hope that this process may help others wishing to develop EBGs.

The authors wish to acknowledge Alvin Lever, MA, FCCP (Honorary), Executive Vice President and CEO of the ACCP, who provided invaluable encouragement and support to consolidate the many efforts in refining the ACCP EBG development process into a published document. The authors also acknowledge the many contributions of current and past members of the ACCP HSP and of the HSP staff, and the contributions of many other ACCP members to the successful evolution of the ACCP EBG development process.