One Billion Neurons on a Desktop Computer

DZone 's Guide to

One Billion Neurons on a Desktop Computer

Check out the latest achievement of the open source Brain Simulator II neuron engine. How does it work and compare to real human brain activity and function?

· Open Source Zone ·
Free Resource

Brain Simulator II: What Is It?

The open-source Brain Simulator II neuron engine has been successfully tested with one billion neurons on a desktop computer comprised completely of off-the-shelf components. From a performance perspective, this system processes more than 2.5 billion synapses per second. For this test, the network included 10 million neurons with 500 synapses per neuron, for a total of 5 billion neurons which could be fully processed in 1,900 milliseconds.

For comparison, the identical program on a four-year-old, four-core CPU can only achieve 0.7 billion synapses per second. The computer in this example included an AMD Ryzen Threadripper 3990X CPU running at 2.9Ghz (not overclocked) and 128 Gigabytes of RAM.

The spiking neural models used are more like biological neurons than traditional AI algorithms and contribute immensely to the efficiency of the program. The model allows for synapses that connect any neuron to any other as well as varying numbers of synapses on each neuron. This flexibility, which is analogous to biological brains, also makes GPU use less effective since GPUs are most efficient when the processing is identical to each neuron. 

How Does This Performance Compare to the Human Brain? 

Energy analysis of the neocortex (the higher-level thinking part of the brain) shows that neurons spike on average only once every six seconds. This means that its 16 billion neurons generate only 2.5 billion spikes per second in total. The spiking neural model only processes neurons which spike in a specific time interval, rather than processing all of them. As a result, it can be thousands of times faster than traditional artificial neural networks.

The number of synapses per neuron, then, is the key to estimating relative performance. While the human brain has thousands of synapses for each neuron, many are either redundant or “synapses-in-waiting”—synapses which could represent memories should the need arise. These extraneous synapses are also not necessary to the simulation since a computer can add new synapses instantly while a brain cannot. Depending on the number of synapses used for the estimate, the current simulation ranges from being more capable than the neocortex to perhaps a thousand-fold slower (easily overcome with a few racks of servers).

To get this performance boost, the neuron engine was translated from C# to a C++ .dll and then extensively profiled and tweaked to get the best effect from CPU caching. This conversion is also the first step toward implementing the neuron engine on a High-Performance Computing (HPC) cluster in which each server processes several million neurons and communications are only needed for synapses that cross the boundaries between machines. In the biological setting, neurons overwhelmingly connect to other neurons within a cluster of 1-10 thousand, suggesting that an HPC cluster approach may be viable for essentially unlimited neuron network size.

Performance tops out with 64 threads, but reducing to 16 threads is only a 25% performance hit as computation is limited by RAM contention.  The innate character of a spiking simulator lets any neuron connect directly to any other. The shorter the average synapse distance is, the greater the performance since the target neuron is more likely to be in the CPU cache.

User Interface and Modules

This performance enhancement applies only to the neuron engine portion of the Brain Simulator II. The user interface and modules which control the creation and editing of the neurons still reside in C#. Many developers believe that development can be faster in C# due to the language’s inherent restriction on direct access to the underlying CPU. In C#, this is an advantage because it is more difficult to make many hard-to-correct programming errors so development can be faster. On the other hand, it is more difficult (or impossible) to squeeze the maximum performance out of the CPU.

The Brain Simulator II is an open-source project to develop AGI which includes the neuron engine and an expanding collection of “modules” which can manipulate the underlying neurons and shortcut calculations. Modules exist for vision, hearing, touch, simulation, imagination, solving mazes, and many others.

Additional information about the neuron engine and its profiling is contained in this video: 

artificial intelligence and machine learning

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}