Columbia Technology | Ventures
  • print
  • email

New Technologies by Columbia Engineers Endow People with Superhuman Search Powers

by Jason Daley, Adam Piore, Preston Lerner, Elizabeth Svoboda; illustrations by Jonathon Rosen

Ailment: Too Much Information

Cure: Mind-Reading Machines

If you have ever felt overwhelmed by a multitude of choices—say, 10,000 items in an online catalog—this brain-boosting invention is for you. The Cortically Coupled Computer Vision (C3Vision) system, designed by engineers Paul Sajda and Shih-Fu Chang of Columbia University and Lucas Parra from the City College of New York, endows people with superhuman search powers, allowing them to find meaningful objects in mountains of images up to 10 times faster than they normally could.

The technology, part of an 18-month, $2.4 million Defense Advanced Research Projects Agency (DARPA) undertaking, relies on electroencephalography (EEG) to detect the cascade of neural firing patterns in your brain when you spot something novel or interesting, even if you’re unaware of it. Sajda offers the example of an intelligence analyst who must rapidly scan satellite photographs or drone footage for suspicious happenings. With C3Vision, he dons an EEG skullcap and starts searching. Whenever he sees something that stands out, his brain exhibits a distinct firing pattern associated with “aha” moments. C3Vision picks that up and applies it to pattern-recognition software, which in turn flips through thousands of other satellite images to cull suspect objects or movements on its own. “The system latches on to individual perceptions and trains the computer to know what the user means by interesting,” Sajda says. “The computer and the brain operate synergistically.”

The Army is interested in using such a mind-machine interface to help soldiers navigate dangerous terrain. A driver might see something peculiar on the roadside. Maybe it is an improvised explosive device. His C3Vision headgear would register the brain waves associated with the suspicious object and inject them into the vehicle’s driving system. When the system sees other things out there that look similar, it would automatically evade them. Likewise, security guards might use such gear to spot suspicious activity on surveillance video.

Sajda envisions the technology eventually improving civilian lives as well, starting with shopping. A miniaturized, wireless version of the device might be used to tag consumer items or even specialty shops that catch your fancy as you walk down a city street. Just a quick glance at a dress in a window, for instance, might elicit a neural firing pattern sufficient to register with the system. A program could then offer up nearby stores selling similar items or shops you might want to investigate. “There’s nothing out there that can really use your subjective preference as a signature to guide you,” Sajda says. “It’s the same type of problem in the analyst world. There’s so much information to explore and digest, how do you make it useful to a person at a given time? We can make it unobtrusive and tag things as you move through your environment.”

Article from Discover Magazine.

To view technologies from Dr. Shih-Fu Chang's lab, please click here.

To view technologies from Dr. Paul Sajda's lab, please click here.