Ruben Coen-Cagli, Ph.D., assistant professor of systems & computational biology and in the Dominick P. Purpura Department Purpura of Neuroscience, has two major grants totaling more than $2.5 million from the National Institutes of Health for basic research in computational neuroscience. The grants support studies on visual processing, neural coding, and machine learning and involve collaborations with Einstein faculty members including Adam Kohn, Ph.D. at Einstein and Pascal Mamassian at Ecole Normale Superieure Paris.

We sat down with Dr. Coen-Cagli to discuss his research, the complex world of visual processing, and what he likes about the booming fields of neuroscience and artificial intelligence.

Q: Your lab focuses on computational neuroscience. Can you explain what that is and describe the kind of research you conduct?

A: Sure. My lab works on developing theories and models of neural coding, which is basically how neurons respond to stimuli. Our research focuses specifically on visual processing. We’re trying to understand how neurons encode and decode information they receive from the eyes, and we’re also studying perception—namely, how the brain integrates all the incoming information and puts it in context so it can guide our interpretation of the world and how we make decisions.

In practical terms, we start by creating two types of models: algorithms that simulate how we think perception works, and computational models based on theories of how neurons will behave. Both types of models are based on data from experiments and our knowledge of the anatomy and function of the visual areas of the brain. We use these models to generate hypotheses about visual perception and visual processing in the brain. Then we do laboratory experiments with collaborators on animal and human subjects. Once the experiments are complete, we use the data to test our theories and our computational models and refine them for the next round. So it’s a closed loop in which theory guides experimental design, and the data from experiments guides theory.

Q: What is visual processing exactly?

A: Visual processing is extracting relevant information from the light patterns on our retina, while discarding irrelevant information. It is something that we humans do seamlessly and effortlessly all the time without realizing we’re doing it. One important aspect of visual processing, which we study in the lab, is integrating information across the visual field. For example, I can very easily segregate the table in front of me from this piece of paper and that pot sitting on top of it. Conversely, I can also integrate into one object all of the parts of the table—legs, all the sections of the surface I can see and that are hidden. This process of analyzing separate elements of a visual scenes—we call them visual features—and grouping them into meaningful objects, is done by large networks of neurons.

Q: So different parts of the brain, different neuronal networks, are responsible for visual processing?

A: Yes. The visual system, as well as all other sensory systems, have sort of a hierarchal organization. When we’re talking about vision, you have the visual sensory organs—the eyeball, the retina—then an intermediate stage, and finally there's the visual cortex. And then the visual cortex is organized in a hierarchy of anatomically distinct and functionally distinct areas.

So the first part of the visual cortex, called the primary visual cortex, is what receives the inputs from the sensory periphery. Neurons in that part of visual cortex process some basic aspects of the image—small parts of the visual field—into very simple features, like orientation. This information is then relayed to the secondary visual cortex, which processes more complex features, like combinations of the features from the previous area. And so on. So there is a cascade or hierarchy of areas.

And as the visual inputs are processed as they progress upward within this hierarchy, the neurons see more and more of the visual space and therefore encode increasingly complex properties. And finally, the neurons at the top encode for entire objects or places or the identity of objects.

At the same time, signals from areas that are higher in the hierarchy propagate down to lower areas. This signal propagation from higher to lower areas is called feedback. Although feedback has been studied much less in the past, interest in it is growing in our scientific community because it has become clear that feedback as well as feedforward play important roles in visual processing. Broadly speaking, we think that feedforward propagation—from lower to higher areas—conveys sensory information, whereas feedback conveys expectations—namely what we expect to perceive in a given context—and that the balance of these two processes is what makes our visual perception so robust and also adaptable.

Q: What is the potential medical use from your research?

A: The overall goal of our basic research is to understand visual processing in a normal brain. That understanding has potential applications for biomedical engineering. If you understand the neural code of the visual cortex you could, for example, improve retinal implants. In addition, our understanding of visual processing could be leveraged to improve computer-vision algorithms, with many potential applications including the use of AI [artificial intelligence] to help in analyzing medical images.

Also, in a number neurodevelopmental and neurodegenerative diseases, there is a strong impairment of certain functions of visual processing. One example is autism spectrum disorder (ASD).

Q: How is visual processing impaired in ASD?

A: One problem seems to be processing what we call contextual information. So, for example, most of us experience visual illusions with certain kinds of visual stimuli due to being influenced by what’s surrounding a target.

For example, there is something known as the Ebbinghaus illusion, where if I show you an image of a circle, its size will appear different depending on whether it is surrounded by smaller circles or by larger circles. [See illustration.] It’s a very strong visual illusion for most of us, but there’s evidence that this and similar illusions are weakened or delayed in autism spectrum disorder as well as in schizophrenia.

In an article published this year, we studied a related perceptual phenomenon, and found that the contextual influence can be reduced by simple visual cues that suggest a segmentation between the target and the context. It would be interesting to study if this flexibility is also affected in ASD. In general, understanding how the processing of visual context differs in those patient populations remains a very active area of investigation.

Q: You’ve mentioned uncertainty and probability as important in visual processing and perception. Can you explain that?

A: Our brains process visual information without our consciously thinking about it, so it’s interesting to learn what the brain and its neurons are actually doing with visual stimuli. As we discussed, the brain performs relatively simple tasks that involve integrating visual context—determining if something is dark or bright, for example, or is large or small, relative to what surrounds it. But the brain must also take into account information not present in the visual stimuli.

For example, the conversion of light to electrical signal in the retina is “stochastic,” meaning that our vision is constantly corrupted by visual “noise” that the brain must deal with. Another example is what we call ambiguity: for example, the world is three-dimensional, but our retina is two-dimensional; so to interpret the 3D shape of objects correctly, we need to use our knowledge of how objects typically look.

Due to this lack of definitive information, our perceptional judgments are often not a hundred percent certain—and having this notion of uncertainty is actually important for guiding our decision-making.

Here’s a very concrete example: if you’re crossing the road on sunny day and you can clearly see there’s no car, you cross. But if it’s foggy and you can’t see very clearly, you’re uncertain about whether a car is coming or not and you’re more careful. So for your own survival, you need to have a representation of uncertainty within visual processing.

Q: How do you measure or determine how the brain calculates probability? What does that experiment look like?

A: That’s a big open question in our field—how neurons could even represent probabilities and uncertainties—and it’s the focus of one of my grants. We and others have theorized that neurons represent probability by varying their activity level even when the visual input is constant. For example, if an image is very clear and there is little uncertainty—think of crossing the road on a sunny day, as we discussed earlier—then neural activity would be more stable. If instead there is lots of uncertainty—the foggy day—then neural activity would vary a lot, and the amount of variation would signal the level of uncertainty.

With my collaborator Adam Kohn, we’ve been trying to test this theory through experiments involving monkeys, in which we present them with the same image many times and measure how much the electrical activity of the neuron changes from presentation to presentation, and how that variation differs for clear versus unclear stimuli. We published initial findings that support the theory in 2019 and recently posted another paper to the pre-print server prior to publication. We are currently testing many other predictions.

Q: The brain is obviously a tremendously complex organ. What do you find most interesting about neuroscience?

A: One of the fascinating things about neuroscience is that you can study it at so many different levels. You can look at the synapses between two neurons. You can go smaller, looking at the molecular and cellular composition of the synapses: what are the chemicals that go back and forth in the synapse? How is the synapse created and destroyed? And then you can go down to the level of genes. And of course you can go up through the hierarchical levels I just described, up to the level of the entire brain and how it generates behavior. So, yes, it’s really complex organ, and it’s really fascinating.