Professor John K. Tsotsos, from the Department of Electrical Engineering and Computer Science at the Lassonde School of Engineering, has focused his career and research on computational and human vision, specifically visual attention. New research has supported a theory he proposed over 30 years ago.
In 1991, when Tsotsos was still at the University of Toronto, he developed a neural network method by which a visual system can focus on items in a scene without eye movement. This ability is termed covert attention and was first described by the German pioneer of experimental psychology Hermann von Helmholtz (ca. 1860). Covert shifts of attention help us perceive day-to-day activities where visual events happen quickly, such as driving, riding a bike or playing a video game, but do not permit enough time for eye movements to focus on every detail. Tsotsos first presented this method at York’s Center for Vision Research conference in June of that year and it was later published as part of his full Selective Tuning theory in the journal Artificial Intelligence in 1995. This computational theory was intended as both effective for machine vision and predictive for human vision.
The theory posed that to achieve covert shifts of attention, our visual cortex – the portion of our brain that processes visual information – must determine the exact locations in our visual surroundings that are of interest. “Imagine a vanilla glazed donut placed in front of you, and you keep your eyes focused on the center of the donut hole,” says Tsotsos. “While your eyes are fixed on the center, the donut is replaced with a similar donut, except this one is decorated with sprinkles. The sprinkles are arranged in eight lines, all pointing in different directions and equally spaced across the top of the donut. Without shifting the focus of your eyes from the center of the donut, we can covertly shift our attention to identify the location of the sprinkle that is pointing vertically.”
The computational mechanism Tsotsos proposed, used to achieve this example of a covert shift of attention, has been recently tested and supported by a long-standing collaboration with neuroscientists in Germany. This pivotal research yielding new evidence was led by Mandy Bartsch and under the direction of Professor Jens-Max Hopf of the Leibniz-Institute for Neurobiology and the Otto-von-Guericke University, in Magdeburg, Germany.
Their work, published under the title “A cortical zoom-in operation underlies covert shifts of visual spatial attention” in the journal Science Advances, revealed that covert shifts of attention are achieved by a progression of focus from high to low levels in our visual cortex hierarchy – just as Tsotsos predicted, over 30 years ago.
This conclusion was reached by investigating activity in the visual cortex during covert shifts of attention, using a series of visual search tasks. Participants were asked to keep their eyes fixed on an image in front of them, while distinguishing colour-defined targets among other items appearing in nearby quadrants.
When participants covertly shifted their attention to detect a target, initial activity was observed in higher levels of the visual cortex, eventually progressing to mid and lower levels. Even when participants shifted their highly concentrated focus from one target to another, activity in the visual cortex would restart from high to low – demonstrating that attentional focus does not simply shift, it restarts a computation beginning in higher levels in our visual cortex. This investigation showed that covert shifts of attention can be attributed to a process that is much like zooming-in on a camera, where the lens begins with a broad target, followed by a more selective focus on finer details in the frame – also known as coarse-to-fine process.
Beyond the contribution of important knowledge to the field of cognitive neuroscience, these research findings can be used to enhance design approaches for advertisements and visual user interfaces for electronic devices. By incorporating knowledge of how human viewers shift their attention covertly, designers can improve the spatial arrangement of important information to maximize what viewers see. In addition, this research may lead to more accurate and efficient machine vision systems that behave more like human vision.