Teaching Computers to “See” the Invisible in Living Cells – NIH Director's Blog

Teaching Computers to “See” the Invisible in Living Cells – NIH Director's Blog

Teaching Computers to “See” the Invisible in Living Cells

Posted on April 24th, 2018 by Dr. Francis Collins

Caption: While analyzing brain cells, a computer program “thinks” about which cellular structure to identify.
Credit: Steven Finkbeiner, University of California, San Francisco and the Gladstone Institutes

For centuries, scientists have trained themselves to look through microscopes and carefully study their structural and molecular features. But those long hours bent over a microscope poring over microscopic images could be less necessary in the years ahead. The job of analyzing cellular features could one day belong to specially trained computers.

In a new study published in the journal Cell, researchers trained computers by feeding them paired sets of fluorescently labeled and unlabeled images of brain tissue millions of times in a row [1]. This allowed the computers to discern patterns in the images, form rules, and apply them to viewing future images. Using this so-called deep learning approach, the researchers demonstrated that the computers not only learned to recognize individual cells, they also developed an almost superhuman ability to identify the cell type and whether a cell was alive or dead. Even more remarkable, the trained computers made all those calls without any need for harsh chemical labels, including fluorescent dyes or stains, which researchers normally require to study cells. In other words, the computers learned to “see” the invisible!

A few years ago, Philip Nelson and Eric Christiansen of Google Accelerated Science, Mountain View, CA were interested in adapting machine learning for a variety of applications. Their hope was to develop deep learning algorithms, common features in smartphones and future self-driving cars, to solve other important problems, including in biology.

But for deep learning to work, vast quantities of training data are required. The Google team heard about Steven Finkbeiner at the University of California, San Francisco (UCSF) and the J. David Gladstone Institutes. He had recently developed with NIH funding the first fully automated robotic microscope called the Brain Bot [2]. The Brain Bot can track thousands of individual cells for months on end, churning out far more information-rich imaging data than his lab could possibly analyze. The Google team asked Finkbeiner to propose some ideas, and a collaboration was born.

In the new study, the researchers first trained a computer by feeding it images of two sets of cells. One was fluorescently labeled to draw out structures that researchers are interested in identifying, and the other was unlabeled. They then repeated the process millions of times to generate learning.

In deep learning, computers look for patterns in data. As they search through the images, pixel by pixel, the computers begin to “see” complex relationships, strengthening some connections in the network and weakening others. As the computer examined other pairs of images, it began to build a network based on patterns of correspondences it recognized between the labeled and unlabeled images. It’s comparable to how our own brain’s neural networks process information, learning to focus on some things but not others.

To put the computer to the test, the researchers presented it with new unlabeled images. They found that the computer’s neural network could identify individual cells by learning to spot its unlabeled cell nucleus. Ultimately, the computer could also tell which cells were alive or dead, even picking out a single dead cell amidst many living ones. That’s something people who look at cells every day can’t reliably do.

The computer could also pick out neurons mixed in with other cell types. It could tell whether a neural extension was an axon (sends outgoing signals) or a dendrite (receives incoming signals), despite the fact that those two cellular appendages look quite similar. As the computer’s knowledge grew, it also required less data to learn a new task.

The findings offer important proof-of-principle that computers can be trained to outperform people in analyzing human cells and other complex images. It also shows there’s clearly more information available in those images than meets the human eye.

Finkbeiner is pursuing many avenues to put his skilled computers to work. For instance, computers might be used to identify which stem cells are best suited for transplantation. They might be capable of diagnosing disease based on images of cells in a dish and sorting out the underlying causes of unexplained cases of diseases, including schizophrenia, Alzheimer’s disease, or Parkinson’s disease. Computers might also be trained to differentiate healthy cells from sick ones, which could prove useful in identifying the most promising new drug candidates.

While there’s still plenty of room to expand and improve the network’s predictive abilities, it’s worth noting that deep learning isn’t limited to imaging data. In fact, the same principles can be applied to any sort of abundant, well-annotated information, including DNA data. Computers could also be trained to look for relationships between data of different kinds, yielding discoveries beyond anything us humans could readily grasp. And they never get tired or complain about the working hours.

References:

[1] In Silico Labeling: Predicting Fluorescent Labels in Unlabeled Images. Christiansen EM, Yang SJ, Ando DM, Javaherian A, Skibinski G, Lipnick S, Mount E, O’Neil A, Shah K, Lee AK, Goyal P, Fedus W, Poplin R, Esteva A, Berndl M, Rubin LL, Nelson P, Finkbeiner S. Cell. 2018 Apr 9. pii: S0092-8674(18)30364-7.

[2] Automated microscope system for determining factors that predict neuronal fate. Arrasate M, Finkbeiner S. Proc Natl Acad Sci U S A. 2005 Mar 8;102(10):3840-3845.

Links:

Steve Finkbeiner (University of California, San Francisco and the J. David Gladstone Institutes)

Google Accelerated Science Team (Google, Inc., Mountain View, CA)

NIH Support: National Institute of Neurological Disorders and Stroke