National Guideline Clearinghouse | Expert Commentaries: A Perspective on the Guidelines International Network and the Institute of Medicine's Proposed Standards for Guideline Development

open here please:
National Guideline Clearinghouse | Expert Commentaries: A Perspective on the Guidelines International Network and the Institute of Medicine's Proposed Standards for Guideline Development

April 8, 2013

 

A Perspective on the Guidelines International Network and the Institute of Medicine's Proposed Standards for Guideline Development

By Amir Qaseem, MD, PhD, MHA

The recent drive to improve health care quality and efficiency has resulted in the rapid growth of clinical practice guideline (CPG) development activity and with it, a variety of development methodologies. CPGs are intended to translate research into practice, to reduce practice variation, and to promote excellence in care. CPGs serve as the practical arm of the evidence-based movement by building on systematic reviews of the best available evidence. Unfortunately guideline developers have been their own worst enemies, and in recent years, guidelines have become entangled in a web of issues related to their development. Concern about the varying quality of guideline development methodologies and recommendations has led to increased frustration among CPG users rather than resolved the clinical problems that the CPGs were meant to address. CPG development involves several steps, each of which may be executed with differing degrees of rigor. Although international collaboration for some projects has resulted in a certain degree of harmonization within the CPG development process, many issues remain. These concerns include variation in the quality of evidence supporting the guidelines, lack of transparency, inadequate disclosure and management of actual or perceived conflicts of interests, concerns about the funding of CPG development, and a lack of agreement on what constitutes a "good" guideline.
The development of CPGs is, of course, a continuous and evolving process. Despite the existence of guideline evaluation tools such as the Appraisal of Guidelines for Research and Evaluation (AGREE) (1), developed to standardize how the quality of CPGs and their development methods are judged, guideline quality remains variable and calls into question the reliability of some apparently authoritative CPGs. To improve the overall quality of CPGs and to promote trustworthy ones, the USA-based Institute of Medicine (IOM) (2) and the international organization Guidelines International Network (G-I-N) (3) have each proposed a set of standards for CPG development. In general, IOM and G-I-N agree on the basic elements essential to developing high quality guidelines. Below is a comparison of the 11 CPG development standards proposed by G-I-N and the eight standards suggested by the IOM.

• A key step in the right direction is that both organizations agree on the definition of the term "guideline," which states that all the constituent clinical recommendations must be based on a systematic review of the relevant literature.
• Both organizations also agree that establishing transparency and disclosing the methods used during all steps of CPG development is essential, from selecting group members and topics to final approval and dissemination.
• Both organizations emphasize the importance of a multidisciplinary guideline development group in order to develop reliable and relevant recommendations.
• Disclosure and management of both financial and non-financial conflicts of interests are identified as vital by both organizations. G-I-N does not make a specific recommendation on how to manage such conflicts. IOM suggests excluding from a guideline development group members who have conflicts, or if included, that only a minority should have conflicts and chair/co-chairs should not have conflicts.
• G-I-N recommends disclosure by the funding organizations regarding financial support for both evidence review and guidelines. IOM also recommends disclosure, and stresses that funders should not have a role in the guideline development process.
• Both organizations agree on the importance of clearly defining the scope of a guideline.
• G-I-N and IOM call for clear, unambiguous guideline recommendations and for the use of specific grading systems to rate the strength of evidence and recommendations.
• IOM is not explicit about the decision making process while G-I-N makes a specific recommendation about the need to establish the decision making process before starting any CPG project.
• Although both G-I-N and IOM recommend external peer review, G-I-N stresses the importance of considering reviewers who are likely to provide comments based on scientific and clinical knowledge rather than on unsubstantiated comments. IOM, on the other hand, calls for review by a full spectrum of relevant stakeholders, which includes the general public.
• G-I-N and IOM agree on the importance of updating guidelines and making them a "living" document. However, G-I-N does not assume that all guidelines necessarily require updating and proposes an alternate route of "sun-setting" a guideline by identifying an expiration date for it.

The IOM and G-I-N standards have been both applauded and criticized. The concern voiced most often is related to the complexity and practicality of implementing the proposed standards, particularly in an environment where funding and resource limitations are a major obstacle for most CPG developers. Many guideline development groups find the lengthy list of G-I-N and IOM standards to be aspirational but infeasible to implement fully, and consider several of the standards especially difficult to adopt. For example, a "full spectrum review by external stakeholders" may prove to be an impossible task, resulting in a never-ending loop of comments, responses, and revisions; the evidence, meanwhile, used to develop guidelines becomes stale and may require updating. Also, systematic reviews can be costly to produce and may not be feasible for many organizations. There may be difficulty finding clinicians who are experts on a topic and also free from any financial or academic conflicts of interest. Experts also question the validity of including one "token" patient in a guideline development group to represent all patients, and express concern over the difficulty of finding a patient who is properly prepared to contribute actively and meaningfully to the evidence-based guideline development process. The IOM standard stating that less than 50% of the guideline development group can have conflicts of interest is a number without any evidence. Benefit of an external review by all relevant stakeholders and its impact on recommendations is an area of debate. Although adoption of these standards represents best practice, it has not been proven that using these standards will lead to the development of better guidelines which are more acceptable, implementable, and result in better clinical outcomes.
Despite all of these criticisms, the proposed standards do not make perfection an enemy of the good. We have to start somewhere. There are certain inherent attributes that will always be associated with the CPG development process. For example, there will always be variation in the individual recommendations between different groups because, despite the same evidence and standardized process, individual groups will assess the evidence, evaluate the outcomes, and weigh the benefits and harms of interventions differently.
So what do we do with these standards? First, it is important to be transparent about the process and to report it. It is also essential to be consistent and clear about what constitutes an evidence-based CPG and what, in fact, is expert opinion. Guideline developers need to agree on the definition of a CPG and to be consistent in the application of that definition. According to G-I-N and IOM, a guideline should always be based on a systematic literature review of evidence (2, 3). We also need to remember that only a very small portion of medicine has evidence from randomized controlled trials, and expert opinion will always have a place in the medical field. Expert opinion, however, should not be referred to as a clinical practice guideline. Once we settle on a definition of what constitutes a guideline, we should tackle the issue of implementing the proposed standards. Some guideline developers have suggested the possibility of "picking and choosing" from the proposed standards and selecting only those standards that are easiest for their organizations to put into practice. However, I would argue against such an approach, as we do not know the weight each standard carries. What we do know is that the adoption of all these standards will be a step in the right direction and will increase the overall quality and reliability of CPGs.
It is important that all guideline developers try to follow these principles. Maybe it is time for organizations to start collaborating and sharing resources, if individual development is not possible. Although recommendations may require tailoring to specific circumstances and health systems, the evidence on which they are based should be universal and thereby provide opportunities for collaboration. Perhaps the organizations that are unable to follow a good process, because of resource constraints or the inability to collaborate, should not develop CPGs at all. Regardless, the process of developing CPGs will continue to evolve. It is important for guideline developers to adapt to this evolution, as survival will favor those who keep up with the changes. It is now time to interact with and to learn from each other, to share our limited resources, and to implement the standards that make up trustworthy guidelines. Agreeing on standards for CPGs is an initial step in the right direction that may one day lead to the realization of the dream of harmonized guidelines.

---

Author
Amir Qaseem, MD, PhD, MHA*
Director, Clinical Policy, American College of Physicians
Chair, Guidelines International Network
Philadelphia, Pennsylvania

*The author wishes to acknowledge Fergus Macbeth, MD; Frode Forland, MD, DPH; and Philip van der Wees, PhD, PT for their review, edits, and comments.

Disclaimer
The views and opinions expressed are those of the author and do not necessarily state or reflect those of the National Guideline Clearinghouse^™ (NGC), the Agency for Healthcare Research and Quality (AHRQ), or its contractor ECRI Institute.
Dr. Qaseem serves as Chair for the Guidelines International Network (G-I-N) North America and is lead author of Guidelines International Network: Toward International Standards for Clinical Practice Guidelines . He declares no financial or personal conflicts of interest with respect to this commentary.
References

1. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Hanna SE, Makarski J; AGREE Next Steps Consortium. Development of the AGREE II, part 2: assessment of validity of items and tools to support application. 2010 Jul 13;182(10):E472-8.
2. Institute of Medicine. Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E, editor(s). Clinical practice guidelines we can trust. Washington (DC): National Academies Press; 2011. 2 p. Also available: http://www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust/Standards.aspx .
3. Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der Wees P. Guidelines International Network: toward international standards for clinical practice guidelines. Annals of Internal Medicine. 2012;156(7):525-31.