Children’s Hospital Launches CLARITY Challenge for Clinical Genome Interpretation

By Kevin Davies

January 23, 2012 | Researchers at Children’s Hospital in Boston have launched the CLARITY Challenge -- a $25,000 competition intended to set and advance standards for clinical genomic analysis and interpretation.
CLARITY stands for Children’s Leadership Award for the Reliable Interpretation and appropriate Transmission of Your genomic information. While the much publicized Archon X PRIZE presented by Medco will offer $10 million in prize money for essentially reaching the $1,000 genome early next year, the CLARITY contest focuses squarely on best practices in clinical genome interpretation and data delivery. The winning team will receive a $25,000 prize underwritten by Children's Hospital
The competition is open to academic and commercial researchers worldwide, with applications due no later than March 1, 2012. For logistical reasons, a maximum of 20 teams will be selected to participate in the competition. The winner of the competition, chosen by a panel of seven judges, will be announced in October 2012.
Industry partners include Life Technologies, which raised the prospect of a $1,000 genome in 2013 with the unveiling of its Ion Proton sequencer last week, and Complete Genomics.
“With the swift decline in the cost of sequencing, the time is rapidly approaching when genomic information will leap from the research bench to the doctor’s office and become a part of everyday care,” said Isaac Kohane, director of Children’s Hospital Boston’s informatics program and one of three competition co-organizers.
“Paramount among the obstacles to true genomic medicine are interpretation and communication. How do we deliver the information encoded in the genome in an understandable way to physicians or patients to help guide better healthcare? Right now, there are no broadly accepted standards for doing so.”
Coming Together
Kohane told Bio-IT World that the idea for launching the contest emerged from the success his group enjoyed in running challenges for i2b2 (informatics for integrating biology & the bedside), focused on making data widely available. “The competitive aspect was nice, adds some spice, but [more importantly] it catalyzes teams coming together. I think that there’s a certain social process around these competitions, creating teams for a purpose that otherwise didn’t exist.”
In addition to helping the patients and their families, Kohane hopes to identify and bring together the best elements of competing pipelines, as he expects that there will be stronger and weaker components for each pipeline. “There’ll be one overall winner, but separate and transparent grading of different components of the pipelines.”
"Clearly, we’re going to need teams” to tackle the challenges of clinical genome interpretation, he says. “One of my favorite publications in 2011 was a paper in Nature Biotechnology from Mike Snyder and colleagues, in which they compared Complete Genomics sequence data to Illumina. They found that there was [only a] 93% concordance for single nucleotide polymorphisms (SNPs), which is terrible! And on copy number variants (CNVs), only 24% concordance. That tells me that every point in the pipeline -- from measurement to assembly to annotation, interpretation and generation of reports – all those points are, to be kind, open for improvement.”
he best way to stimulate “a public and transparent improvement of that pipeline” is to compare them side by side, he says, not only for the Consumer Reports-style value but also “to promote best practices in different pipelines, so they can be adopted and shared.”
Three pediatric patients at the Manton Center have been selected, including two with neuromuscular disorders. In each case, doctors strongly suspect a genetic basis for the childrens’ conditions, but “despite the best efforts of clinical geneticists at several sites, the genetic lesion has yet to be identified,” says Kohane.
Manton Center director Alan Beggs recruited the three families taking part. “Their response ranged from enthusiastic and very excited to cautiously optimistic,” says Kohane. Getting IRB approval from Children’s Hospital was tough, he says, in part because of the data sharing requirements.
Each patient and their parents will be sequenced. Life Technologies will sequence the exomes of each volunteer, while Complete Genomics will do whole genome sequencing.
“The project underscores Complete Genomics’ commitment to, and the industry’s path towards, using high quality, accurate genomic information from whole-genome sequencing (WGS) to improve patient care,” said Complete Genomics CEO Clifford Reid. “Through this CLARITY Challenge, we anticipate the discovery of the genetic basis of the children’s unknown disorders and also the creation of best practices for interpreting and presenting actionable results to physicians, patients and their families.”
Judge and Jury
The seven judges were chosen for their expertise in different parts of the pipeline. They are: Russ Altman (Stanford), Elaine Lyon (ARUP Labs), Joseph Majzoub (Children’s Hospital), David McCallie Jr (Cerner Medical Informatics Institute), Peter Neupert (Microsoft Health Solutions Group), Peter Szolovits (MIT) and Hunt Willard (Duke University).
Among the questions they will be asking: Is the assembly done well? Are the annotations credible? Does the report look readable? Does it make clinical sense? “Not inconsequentially, for a medical director, can they understand the link between the report and the original data? For example, the details of a CNV algorithm are not always transparent to the clinical end user,” says Kohane.
Kohane and colleagues held a couple of workshops in Boston over the past 18 months gauging the interest of community experts in holding a competition. “There have been other contests for various pieces of the pipelines, but we wanted to see if it made sense to bring it together in a true clinical application,” says Kohane. “Did anybody buy the idea that there must be a community of collaborators?”
Kohane praises the progress made by a handful of genomics groups in annotating medical genomes, including the work of the team at the Medical College of Wisconsin. “They all did a terrific job, heroic efforts – from the publications, it seems evident they’ve used lots of computation and inspiration and knowledge of the disease(s). What’s not clear is that the effort could be generalized, across a large number of diseases. We’ve seen panels of experts working hard to help one patient using everything they had, where much of it is in their head.”
“This has to be reduced to clinical practice -- I order a test, and get back a report that a clinician finds intelligible. I don’t need a number of world-leading scientists scratching their head.”
Children’s Hospital staff says it is the first time that a healthcare institution has issued a broad call for the development of clear, consistent ways of applying genomic insights to patient care. But they note there are numerous pitfalls to be overcome by the contesting teams, including:

Inconsistent or non-specific sequencing results and non-interoperable processes
Conflicting gene variant annotations and classification
A hodge-podge of non-standardized databases
Lack of standards concerning individual privacy and data access
Resulting reports that are not clear or useful to doctors, genetic counselors and patients