Assuring the quality of next-generation sequencing in clinical laboratory practice

Journal name:: Nature Biotechnology
Volume:: 30,
Pages:: 1033–1036
Year published:: (2012)
DOI:: doi:10.1038/nbt.2403

Published online: 08 November 2012

To the Editor

We direct your readers' attention to the principles and guidelines (Supplementary Guidelines) developed by the Next-generation Sequencing: Standardization of Clinical Testing (Nex-StoCT) workgroup. These guidelines represent initial steps to ensure that results from tests based on next-generation sequencing (NGS) are reliable and useful for clinical decision making. The US Centers for Disease Control and Prevention (CDC) convened this national workgroup, which collaborated to define platform-independent approaches for establishing technical process elements of a quality management system (QMS) to assure the analytical validity and compliance of NGS tests with existing regulatory and professional quality standards. The workgroup identified and addressed gaps in quality practices that could compromise the quality of both clinical laboratory services and translational efforts needed to advance the implementation and utility of NGS in clinical settings.
The workgroup was composed of experts with knowledge of and experience with NGS and included clinical laboratory directors, clinicians, platform and software developers and informaticians, as well as individuals actively engaged in NGS guideline development from accreditation bodies and professional organizations. Representatives from US government agencies also participated.
These guidelines address four topics that are components of quality management in a clinical environment: (i) test validation, (ii) quality control (QC) procedures to assure and maintain accurate test results, (iii) the independent assessment of test performance through proficiency testing (PT) or alternative approaches and (iv) reference materials (RMs). Discussions were limited to the analytic and informatics processes required for accurate variant calling. The workgroup did not address how variants are prioritized, interpreted or reported.
The workgroup recommendations are summarized in Table 1. Although the workgroup focused on detection of DNA sequence variations associated with heritable human disorders, many of the principles and recommendations described are also relevant to the application of NGS to other areas of laboratory medicine, including the diagnosis, prognosis and treatment of cancer and infectious-disease testing.

Table 1: Selected workgroup recommendations for establishing NGS test systems for clinical use

Full table

Validation is the process of establishing analytical performance specifications for a clinical test system developed in house to confirm that the system is suitable for its intended use¹. During the validation process, the laboratory must demonstrate that the assay functions as expected and provides reliable results. The workgroup considered the requirements of the Clinical Laboratory Improvement Amendments (CLIA) and provided recommendations for validation of clinical NGS tests. The validation process can be divided into three stages: platform, test-specific and informatics pipeline validation (Supplementary Fig. 1). Platform validation provides evidence that the assay can deliver reliable sequence data within the regions of the genome targeted for analysis. Test-specific validation demonstrates that the assay can detect clinically important sequence variations for the intended application. Validation of the informatics pipeline establishes the software settings necessary to ensure that the test can reliably provide accurate sequence data and detect variations. Although each stage of validation is considered separately, they are interdependent. Validation requires application-specific considerations for whether the test targets a gene panel, the exome or the whole genome, as well as the types of sequence variations that are detected.
In the United States, diagnostic tests that are provided to clinical laboratories are regulated by the US Food and Drug Administration (FDA). To date, no NGS technologies have been approved or cleared by the FDA. These tests are developed in house as laboratory-developed tests and are regulated under CLIA². The CLIA regulations define evaluation of analytical reliability and limitations and require laboratories to establish specifications for performance characteristics for each test system developed in house. The characteristics evaluated to establish the analytical validity of test results include accuracy, precision, analytical sensitivity, analytical specificity, reportable range, reference range or reference intervals, and other relevant performance metrics. Laboratories in the United States and other countries may also comply with the QMS standards described in ISO 15189 (ref. 3). The performance characteristics defined in CLIA² and professional guidance documents⁴ (ACMG standards, guidelines, and policies available at; CAP NGS checklist available to subscribers at http://www.cap.org/) do not readily translate to NGS testing practices owing to the complexity of the technology and the informatics analyses required for large-scale genome analyses. Therefore, the workgroup adapted the definitions of these performance characteristics to better fit the use of NGS in the clinical laboratory (Table 2). A comparison between the CLIA definitions and those developed by the workgroup is presented in Supplementary Table 1, and the unique metrics for NGS that laboratories should establish and monitor to assure high-quality analytical results are presented in Supplementary Table 2. For example, the depth of coverage, or the number of independent reads assessed when making a base call, is a crucial metric for establishing the accuracy, analytical sensitivity and analytical specificity of an NGS test. Owing to the high costs and extensive data analyses required for these tests, it is challenging to establish the precision (e.g., repeatability of testing replicates) of an NGS assay by determining concordance of sequencing results among a large number of samples. The workgroup proposed an alternative approach in which additional metrics such as the depth and uniformity of sequencing coverage would be incorporated to limit the number of samples required to establish precision. The workgroup redefined 'reportable range' and 'reference range' in terms of the qualitative nature of these DNA test results. This guideline classifies reportable range as the region of the genome from which sequence of an acceptable quality can be derived by the laboratory test, and reference range as the spectrum of nonpathogenic base changes observed in a population. Sequence variations outside this spectrum could be disease associated, but further investigation may be necessary to confirm disease association. The determination of reference range is problematic because the spectrum of sequence variations that can be defined as 'normal' or deleterious will vary across individuals and populations.