Much hope has been placed on therapies that play on the same mutations that make cells cancerous — but selecting the right drugs may be more complicated than expected.
SAM OGDEN/SCIENCE PHOTO LIBRARY
The drive to exploit the particular genetics of a patient's tumours to treat them is being complicated by a new study that found flaws in how such drugs are being screened.
Researchers led by computational biologist John Quackenbush at the Dana-Farber Cancer Institute in Boston, Massachusetts, compared data from two studies, published in Nature1, 2 last year, which tested whether drugs were effective against different cancer cell lines grown in culture. Of all the combinations tested, 15 drugs and 471 cell lines were examined in both studies. Both studies also analysed the genetic characteristics of the cells, such as mutations in their DNA, and gene expression — a measurement of how much RNA corresponding to each gene was produced by the cells.
When they were published, the studies were hailed as an important step in the burgeoning field of targeted cancer medicine, which aims to find drugs that attack the precise genetic features of a tumour. This field has exploded during the past few years as drug companies have developed drugs tailored to these genetic features. For instance, vemurafenib, approved by US regulators in 2011, attacks cells harbouring a specfic mutation in a gene called BRAF that is abnormally active in some forms of skin cancer.
But in a new Nature paper3, Quackenbush and his colleagues say today that the data from the original studies may not be as useful as scientists had hoped, because the studies give conflicting answers.
“This is a broad cautionary tale,” says Quackenbush. “We need to be careful about how we define phenotypes, such as whether a patient is likely to respond to a drug or have an adverse event, because if we don’t do it well, we’re not going to have good tools for advancing personalized medicine.”
The two studies used different methods to decide whether a cancer cell line was susceptible to a particular drug. Quackenbush and his co-workers analysed the results using several techniques, and found that only two or three of the 15 drugs behaved similarly in both studies. In some cases, one study classified a cell line as susceptible to a drug but the other gave the opposite result.

Genetic agreement

The authors of the studies do not accept that the results are in conflict. “It’s not at all clear that the data don’t agree,” says Levi Garraway, a cancer geneticist at Dana-Farber, who co-led one of the studies1. He notes that both papers correctly reported that cells with known genetic vulnerabilities to certain drugs were susceptible to those drugs — for instance, they both identified cell lines with BRAF mutations as sensitive to inhibitors of the protein made by the BRAF gene, and to inhibitors of MEK, the protein it activates. To Garraway, this indicates that both screening methods were capable of finding useful drug targets.

Related stories

More related stories
Still, Quackenbush's analysis is sparking discussion about how researchers in cancer-drug development and personalized medicine should treat large-scale drug-screening data.
Glenn Begley, chief scientific officer of Tetralogic Pharmaceuticals in Malvern, Pennsylvania, who has called attention to reproducibility problems in drug development, says that the paper offers “an important lesson.”
“If you are going to use these results to underpin drug-development programmes, you need to make sure they are solid, and here is an example where there is surprising discordance between different data sets,” he says.

Standard problems

Lillian Siu, a senior physician at Princess Margaret Cancer Centre in Toronto, Canada, who performs clinical trials of cancer drugs based on data such as cell line screening studies, says, “It does raise awareness that when we read the preclinical data, we should be more aware of the fact that there are inconsistencies that occur that could lead to different conclusions.”
Quackenbush and his colleagues say that there is a need for different groups to work together to devise standards for experiments that aim to test drug efficacy through large-scale cell-line screening. His group found that genetic data from the two studies mostly agreed — for instance, both found similar gene expression data — demonstrating how an effort over the past decade to set standards for genetic-data collection has boosted the reliability of those data, he says.
Cyril Benes of the Massachusetts General Hospital in Boston, co-leader of the Cancer Genome Project paper — the second of the two papers analyzed by Quackenbush — says that his project and Garraway's are already discussing how the data collected by their groups might be made more useful to other researchers. But, he says, “These are very large  and complicated efforts. I don't think we can afford to put all these programs of discovery on hold while we do a rethink on methods and standardization.”
Besides, say other researchers, such as Gordon Mills, chair of systems biology at the MD Anderson Cancer Center in Houston, Texas, setting standards for large-scale drug-screening studies will be more complicated than it sounds, because it is not clear which assays best predict which drugs will work in humans.
“This paper is formulating a major problem, and there is no obvious solution for it at this point in time,” he says.