Researchers in computer science and neuroscience have been steadily working to uncover the core design principles underlying intelligent behavior, and inventing key technologies needed to build machines that emulate it. Now, with a recent discovery at Hewlett-Packard Labs, the field is poised to make a massive leap forward by being able to finally build large, brain-like systems running on inexpensive and widely available hardware.
In May 2008, USA’s Defense Advanced Research Projects Agency (DARPA), with a track record in promoting high risk, high reward projects (such as the precursor of the Internet), jump-started the process via the Systems of Neuromorphic Adaptive Plastic Scalable Electronics (SyNAPSE) initiative. The goal of this research program is to create electronic neuromorphic machine technology that is scalable to biological levels. SyNAPSE is a complex, multi-faceted project, but traces its roots to two fundamental problems.

First, classical AI algorithms generally perform poorly in the complex, real-world environments where biological agents thrive. Second, traditional microprocessors are extremely inefficient at executing highly distributed, data-intensive algorithms, whereas biological computation is highly distributed and deeply data-intensive. SyNAPSE seeks develop a new generation of nanotechnology necessary for the efficient implementation of algorithms that more closely mimic biological intelligence. DARPA has awarded funds to three prime contractors: HP, HRL and IBM. Before the launch of the SyNAPSE project, HP made a key advance towards the creation of compact, low-power hardware that could support biological computation: the discovery of the memristor. The concept of the “memristor” wasn’t new, having been predicted by a symmetry argument by Leon Chua in 1971. Chua noticed that the three passive circuit elements, the resistor, inductor, and capacitor, ought to be part of a family of four. This fourth device, which Chua called the memristor, would behave like a resistor with a conductance that changed as a function of its internal state and the voltage applied. In other words, it will behave like a memory.

2
Chua’s work was by-and-large ignored, though, until Greg Snider at HP Labs realized that a strange nanoscale device he was working on exhibited behavior predicted by Chua. This behavior, called a pinched hysteresis loop, had shown up periodically in the nanotechnology literature going back many years. Greg, however, was the first person to find the connection between the data and Chua’s theory. This discovery was crucial for the future of neuromorphic technology because memristors are the first memory technology with high enough power efficiency and density to rival biological computation.
The new HP memristor-based neuromorphic chip is a critical step because it brings data close to computation, much as biological systems do. The architecture is closer to a conventional massively-multicore processor than the neuromorphic processors developed by other groups, but with a very high density memristive memory layered directly on top. Each core has direct access to its own large bank of memory, which dramatically cuts wire length and thus, power consumption.
This architecture is possible because memristors are passive components, compatible with conventional manufacturing processes, and extremely small. The passive property is what enables the miniscule power consumption. Memristors store information through physical changes to the device which require no power to maintain, so power is only needed when a memristor must be updated to a new value. This property is described by the term “non-volatile”. Flash memory circuits are also non-volatile, but they are vastly larger and use significantly more power. Because this memory is compatible with standard manufacturing processes, it can be layered directly on top of a conventional processor. This yields another massive drop in power consumption, because data need not be shuttled over long distances. Finally, because a memristive memory cell consists of nothing more than a single memristor sandwiched between two nanowires, memristive memories can be manufactured at extremely high density. The SyNAPSE project aims to support the manufacture of billions of synapses per square centimeter, but trillions or more are possible in the near future thanks to improvement in the technology and increase in the number of layers of memristors to be deposited.

Taken together, these factors wipe away one of the fundamental limiting factors in prior generations of neuromorphic hardware: the lack of scalability. With a conventional hardware manufacturing process, the amount of surface area dedicated to simulating a neuron is roughly the same as the surface area needed to simulate a synapse. In biological systems, the ratio in size between neurons and synapses averages ten thousand to one, and can range from a few to one all the way up to a hundred thousand to one. Memristors are sufficiently small compared with conventional components that the surface area required to simulate a synapse drops to a small fraction of the area needed to simulate a neuron — a ratio much closer to biology. This cuts power by dramatically reducing the switching and signaling overhead, enabling much higher scalability and computing density. A hardware device based on memristors capable of simulating about the same number of neurons and synapses of a large mammal would take up the volume of a shoebox, with a power consumption of about 1kW. As much power as a common espresso machine needs.
Even though the HP technology has provided a huge advance in the field of hardware compatible with neural computing, the ability to build intelligent software to run on these devices is a big challenge to the SyNAPSE program. HP is working with a team of researchers in the department of Cognitive and Neural Systems at Boston University (BU) to solve this challenge. The BU team is uniquely poised to offer such a solution because of its multidisciplinary approach to building intelligent machines. Such a project requires expertise from fields such as neuroscience, mathematics, engineering, psychology, and robotics. The Center of Excellence for Learning in Education, Science, and Technology (CELEST), a National Science Foundation (NSF) Science of Learning Center founded in 2004, — of which the BU team is an integral part — is attempting just that. CELEST’s research approach develops a new paradigm to simultaneously study and understand brains and behavior to apply insight from computational modeling to building intelligent machines.

3
The innovative memristor-based device, manufactured at HP and equipped with neural models designed, developed and implemented at BU, will dramatically lower the barrier to studying the brain and simulating large-scale brain-inspired computing systems, radically accelerating progress in basic neuroscience research, and spin-off technological applications. The new technology is also fundamentally inexpensive and small — so small that the equivalent of a fairly large network of cortical cells could be deployed in a cell phone. A major step will be to simulate the behavior of a fairly complex brain powering an artificial organism in a virtual environment. Boston University and HP are currently designing the perceptual, navigation, and emotional systems that will emulate some basic rodent behavior on hardware. The simulated nervous system, initially implemented on racks of conventional computers and then transferred to a number of smaller chips, will allow the animat to learn, via plastic changes in synaptic connections among neurons, how to interact intelligently with its environment: searching for food, following learned paths, avoiding punishment and predators, and later competing with other animats for resources.

author info:

Massimiliano Versace, PhD

Dr. Versace is a Senior Research Scientist at the Department of Cognitive and Neural Systems at Boston University, Director of Neuromorphics Lab, and co-Director of Technology Outreach at the NSF Science of Learning Center CELEST: Center of Excellence for Learning in Education, Science, and Technology. He is a co-PI of the Boston University subcontract with Hewlett Packard in the DARPA Systems of Neuromorphics Adaptive Plastic Scalable Electronics (SyNAPSE) project. He earned his PhD in Cognitive and Neural Systems from Boston