If computers could see and recognize objects the way people do, at least one of life's many minor frustrations would disappear — problems at the self-checkout grocery lane. With a more perceptive computer, there would be no more carefully weighing and placing the apples or six-pack of beer in the grocery bag only to have the machine repeatedly tell you "unidentified item placed in bag" until you have to call for a human checker to rectify the situation.

A visually competent computer would recognize the apples or beer and know you put them in the bag. That's just one of many way a computer engineered to "see" like a human might come in handy.

Here's another practical use of such technology — searching for that whatchamacallit or certain item, which you can't tell a search engine how to find.

"If I wanted to search for a particular sexy old car or an elegant dinner jacket, wouldn't it be great if I could take a picture of what I wanted, upload it and tell Google to find what's in the picture?" said Garrett Kenyon, a neurophysicist at Los Alamos National Laboratory who is studying the way the brain processes visual information. "If we could help computers understand what underlies the 'aha!' moment when we recognize something, we would be able to communicate better with it about what we were looking for visually. The computer would be able to understand."

Work by Kenyon and colleagues from LANL, Chatham University and Emory University will shed a little more light on how humans visually recognize objects, another step closer to a more perceptive computer. The team's work was published in the fall in the Public Library of Science (PLoS) Computational Biology journal.

They were testing how long people needed to see an image before they could process and identify the shape. They mapped both the time and the portions of the human cerebral cortex involved in recognizing visual stimuli.

How the brain sees

Scientists know a lot about the mechanisms and organs involved in human sight, but still don't understand all the finer points of how our brains actually process what we see. Scientists have mapped out four parts, or lobes, of the cerebral cortex and assigned some basic functions to each — movement and recognition in the parietal lobe, visual processing in the occipital lobe, for example. But the borders between the lobes and their functions are blurred. Each cortical lobe has neurons and cortical columns that communicate and control some aspect of brain function. Scientists are still trying to figure out exactly how the pieces collaborate, but they're pretty sure the basic process is shared by all mammals. "We're interested in how neural circuits compute," Kenyon said.

Kenyon likens the brain to a "very, very sophisticated car engine. We know the parts of the machine but we don't know [everything about] how they work together. We certainly don't know how to build one."

"We understand the gross anatomy of vision," he said. "We understand the scaffolding, the gross architecture of how vision flows to the cerebral cortex. But different parts of the brain process different parts of sight. Some parts process shapes and forms. Some process color. Some process motion. Somehow these separate pathways come together. We don't know the ultimate answer to what's different between when we see something and when we don't."

In other words, to know the apple is in the bag at the check-out stand, the human brain has to process the shape and color of the fruit, the hand, and the act of putting it in a bag. What humans and other mammals, even babies, can do with little conscious thought, a computer can't do without specific instructions.

Building a brain

Once scientists can figure out how the cerebral cortex sees and recognizes objects, they are well on their way to building more cognitive computers, ones that can more closely mimic the way a human brain "sees" and interprets visual stimulation. Their goal is to combine the speed at which a computer can zoom through mathematical computations with the human brain's ability to recognize and interpret constantly changing visual data. Processing visual information involves thousands of computations, all by a mass of neurons snugly boxed in a container about the size of a half-gallon pitcher.

The effort to reverse engineer a human brain has been going on for years. In 2007, a team at IBM Almaden Research Center set out to build a brain akin to a human, only with hardware "neurons" replacing the mushy gray matter that makes up the cerebral cortex.

Jeff Hawkins, who developed the Palm Pilot and the Treo smartphone, is now working on another project with long-time business associate Donna Dubinsky in their company, Numenta. They are developing a program for extracting information from data by mimicking the way the neocortex processes visual information. The private beta version of the program is due out soon.

The 'aha' moment

The research by Kenyon and colleagues focused on a "speed of sight" task, which tests the amount of time a person needs to see an image in order to correctly identify it. The speed of sight measurements were combined with high-performance computer simulations of cortical circuits, using a neural simulation toolbox developed by LANL called PetaVision.

"You would always know a photograph of a mouse if you had adequate time to look at the image," Kenyon said. "But if I only let you look at an image for a fraction of a second, no longer than a single frame of a motion picture and then I mask the image, for many of those images you won't have a sense of what you saw. If I ask you was there a mouse, or a face, or a car, you just won't know."

So, being human, we guess at the answer. "Neuroscientists keep track of the guesses. At the same time, they will measure cortical activity. They'll try to see what is it that's different when you got it right, when you grasped the image, when it clicked."

If the image flashed for 20 milliseconds, the test subjects always had to guess at what they saw. At 200 milliseconds, they almost always got it right. Somewhere in between was the moment of recognition.

In that precise moment when we recognize what we're looking at — that millisecond of time when our brains say 'aha' — some part of the cortex is prancing about with excitement, ever so briefly. Neuroscientists are honing in on which neurons, circuits and fibers are celebrating the 'aha.'

"We don't know yet where in the brain that recognition occurs or what it is about the activity in the brain," Kenyon said. "What's different in your brain when you 'saw' and recognized the object as opposed to when you didn't?"

"The truth is ... we cannot [yet] write a computer program or an application that mimics the visual processing ability of a mammalian species," Kenyon said. "Computers are blind."

'A game changer'

Kenyon figures a computer that could see, recognize, perceive and interpret visual data the way a human brain can, "would be a game changer."

Kenyon said there's two camps to solving the question of how the brain sees. One camp says ignore biological systems and tackle the problem purely from a mathematical engineering approach. "I'm in the other camp," Kenyon said. "There's only one example we know of in the universe of a system that can see, and those are biological systems. So our approach is to study biological systems, figure out how they do it. Where is the magic in the neural circuit that gives rise to recognition?"

Contact Staci Matlock at 986-3055 or smatlock@sfnewmexican.com.