SpiNNaker
The SpiNNaker 1Million machine is a novel neuromorphic (i.e., brain-like) computing system made up of over a million low power ARM computing cores, used to simulate neurons, assembled into SpiNNaker chips each containing up to 18 such cores along with local and chip-shared memory.
These chips are connected using a custom designed network architecture which connects each chip to 6 adjacent chips, forming a mesh of highly interconnected chips and enabling fast brain-like communication of “spikes”. This allows the simulation of potentially very large brain-like networks of spiking neurons; these neurons are considered to be more biologically accurate than those in traditional neural networks, and so are more suited to simulating biologically relevant networks.
This made the inclusion of the SpiNNaker machine in the Human Brain Project an obvious choice, where it is used to demonstrate the benefits of neuromorphic computing alongside the BrainScaleS system, an analogue computing system that emulates spiking neurons using physical electronic components. The SpiNNaker hardware was already available before the start of the HBP, and EPSRC has funded the construction of the million-core machine. Development of the software and the construction of the SpiNNaker 1Million machine was done simultaneously at the start of the HBP, with all members of the project team, including those in Research IT, helping with the construction. This was finally completed in 2018, when the machine was presented to the world at the official switch on event[1].
Research IT Involvement
At the start of the Human Brain Project the software team consisted of just two people, one of whom was Dr. Andrew Rowley who was seconded to the project for its 10-year duration from the Research Software Engineering team in Research IT. This made it easy for the project to get going without having to recruit too many people and provided a known-skilled senior software engineer to the project to help to lead the team in the development effort, from providing the basic software architecture, to enabling the running of the software and services required to integrate with the Human Brain project.
In the later stages of the project the scale of the software meant that two software engineers were insufficient to develop, maintain and run the software so additional resource was requested, with some being found from recruitment and Dr. Donal Fellows being added to the team from Research IT for the remaining 7 years of the project. This again meant that we had additional resource with proven skills in software engineering to add to the team.
Donal enabled much of the testing framework of the project which has led to a huge increase in the stability of the software over the years, and which has then helped to bring the software to Technology Readiness Level 8 by the end of the project. He also developed a version of the SpiNNaker allocation service that allows remote proxy connections to the machine, enabling it to be used from any location in the world.
Within the team we also recruited Dr. Andrew Gait (previously in Imaging Sciences), who in addition to being a valued member of the software development team, also helped to support users of the machine including providing support for multiple postdoctoral projects across the SpiNNaker group and indeed over the world. Andrew has now gone on to join the Research IT team as a Research Software Engineer with the others.
HBP Software
The HBP funded the development of the high-level software for the SpiNNaker machine, which allows the users to describe their networks using the python based PyNN description language, which the software then compiles and executes on the machine, and extracts and converts the results after completion. This software is written mainly in Python, as this is then directly compatible with PyNN.
However, the code that executes on the SpiNNaker machine itself is written in C code. This is compiled for the ARM cores using gcc, but it is also then converted into a more compact form that can be extracted easily when loaded on to the cores of the machine. This requires that the C code is what is known as “bare metal”, meaning that it does not have a nice Linux kernel underneath to protect the user from themselves. This means that the code does not benefit from memory protection – meaning that errant code can even overwrite the instruction memory itself or overwrite structures in the SDRAM used by other cores! This requires a certain level of skill from the developers both in construction and in debugging, a skill set that is now integrated into the Research IT team.
The SpiNNaker allocation service was written using Java, as this needed a certain amount of multi-threading that is not easy to come by in Python. It was also found useful to convert some of the Python code into Java to enable faster loading of data on to and reading back of results from the machine, again making use of the easy parallelism provided by this language.
Looking to the Future
The project has achieved many things over its duration, all with major contributions from the software [2][3] built with the help of Research IT. The software is now deployed on the HBP EBRAINS Jupyter portal [4] which uses the proxy service to allow any EBRAINS user in the world to use the machine. It has also been integrated with the HBP Neuro-robotics environment, enabling the NRP robots to use SpiNNaker neural network “brains”.
The software has allowed the first neuromorphic simulation of a 1mm-squared area of brain cortex and was the first platform to run this in real-time [5] (meaning that the simulation takes as long to run as the simulated time span) and has since gone on to run a multi-area cortical model which requires nearly half of the million-core machine. The software has also been enhanced to support high-bandwidth input required to use a “silicon retina”[6], a device that works like a biological retina in the eye to detect changes in the intensity of light to produce spikes. This can enable visual input into the neural network, and the software has also been extended to support convolutional neural networks of the sort used in many of today’s AI applications.
Finally, the software has also enabled the running of a new algorithm for classification on SpiNNaker, known as BitBrain [7], which is now the subject of a patent.
Although the Human Brain Project has now finished, work still continues on SpiNNaker, and more projects are upcoming and to be staffed with Research Software Engineers from Research IT. The next generation of SpiNNaker 2 chips [8] has now been co-developed with TU Dresden, and the software now needs to be adapted to work with the new chips. As more projects become funded, Research IT is the obvious first place to look for people to work on the software, with so much experience and expertise available in the team. The SpiNNaker project looks forward to continuing this relationship for the foreseeable future.
[2] https://doi.org/10.3389/fnins.2019.00231
[3] https://doi.org/10.3389/fnins.2018.00816
[5] https://doi.org/10.1098/rsta.2019.0160
[6] https://doi.org/10.1145/3589737.3605969