Report on the Evidence Regarding Off-Label Indications for Targeted Therapies used in Cancer Treatment (Disposition of Comments)

Report on the Evidence Regarding Off-Label Indications for Targeted Therapies used in Cancer Treatment
Disposition of Comments
Project ID: CANT1106

Overview of Peer and Public Review Comments and their Disposition
Each comment submitted by peer and public reviewers has been carefully considered by the authors of the report, and has been responded to within two separate tables. At the outset, additionally, we would like to summarize the main themes raised by respondents and provide an overview response.

The majority of comments agreed with the primary conclusions of the report, that: systematic reviews are not always a feasible and/or reliable way to answer questions regarding clinical effectiveness of new therapies; data relevant to efficacy of any given drug vary greatly in quantity and quality, making it difficult to combine data or compare results across studies, and therefore difficult to draw conclusions; timeliness, in fields where the evidence is rapidly emerging, poses a major challenge when using traditional systematic review methodology, and; in the absence of better sources or methodologies, it is important to work with the existing approach to evidence synthesis/review (i.e., the compendia) so that clinicians have some source, albeit imperfect, to guide them in clinical decision-making. These comments are appreciated, as they demonstrate a general agreement that the current system of evidence synthesis and review warrants improvement. This recognition, in itself, is an important first step toward change.

A second category of comments expressed the concern that results of the 19 systematic reviews forming the body of this technology assessment should not be used to make either reimbursement or clinical decisions. A critical outcome of this work was a realistic appraisal of the feasibility of current methods of evidence review in the present-day research environment. The 19 separate systematic reviews comprising this report could be viewed as sub-studies which, taken together, powerfully illustrate the challenges and obstacles inherent in attempting to perform an up-to-date and comprehensive review of the evidence in a field where the research pace is rapid, studies vary considerably in design and quality, and clinical decisions hinging on their results are urgent. In order to conduct the systematic reviews, we needed to establish an arbitrary cut-off date for study inclusion (the last searches were conducted on September 14, 2007). As a result, the report is admittedly neither complete nor entirely up-to-date; of necessity, it had to omit studies that were conducted, published, or indexed in MEDLINE® after the cut-off date. Thus, we do not offer the results in this report as definitive for use as the basis of decision-making. Instead, the report is intended to inform a national discussion regarding how best to evaluate and synthesize data so that decision-makers have access to timely evidence that has been reviewed for quality.

A third category of responses addressed the evidence we included, and our manner of presenting the results of the 19 systematic reviews. Reviewers suggested numerous additional references and studies for inclusion in the systematic reviews. We reviewed each of the 146 suggested citations and identified five that met inclusion criteria but had not been included in our reviews. We evaluated all five references and determined that including them in the revised report would not alter our conclusions and would add little or no informative evidence to the report. Finally, some respondents suggested that we present the evidence in the format of more traditional summary evidence tables. We understand the desire for a familiar structure; however, the disparate nature of the data in many cases made it impossible to combine them, as required to create a comprehensive table.

Select for Table 1: Invited Peer Reviewer Comments (PDF file, 213 KB). PDF Help.
Select for Table 2: Public Review Comments (PDF file, 886 KB). PDF Help.

open here to see the full-text:
Report on the Evidence Regarding Off-Label Indications for Targeted Therapies used in Cancer Treatment (Disposition of Comments)