colanet
Continual Learning with Columnar Spiking Neural Networks
Larionov, Denis, Bazenkov, Nikolay, Kiselev, Mikhail
Continual learning is a key feature of biological neural systems, but artificial neural networks often suffer from catastrophic forgetting. Instead of backpropagation, biologically plausible learning algorithms may enable stable continual learning. This study proposes columnar-organized spiking neural networks (SNNs) with local learning rules for continual learning and catastrophic forgetting. Using CoLaNET (Columnar Layered Network), we show that its microcolumns adapt most efficiently to new tasks when they lack shared structure with prior learning. We demonstrate how CoLaNET hyperparameters govern the trade-off between retaining old knowledge (stability) and acquiring new information (plasticity). We evaluate CoLaNET on two benchmarks: Permuted MNIST (ten sequential pixel-permuted tasks) and a two-task MNIST/EMNIST setup. Our model learns ten sequential tasks effectively, maintaining 92% accuracy on each. It shows low forgetting, with only 4% performance degradation on the first task after training on nine subsequent tasks.
- Asia > Russia (0.05)
- Europe > Russia > Central Federal District > Moscow Oblast > Moscow (0.04)
- Europe > Russia > Volga Federal District > Chuvash Republic > Cheboksary (0.04)
Spiffy: Efficient Implementation of CoLaNET for Raspberry Pi
Derzhavin, Andrey, Larionov, Denis
This paper presents a lightweight software-based approach for running spiking neural networks (SNNs) without relying on specialized neuromorphic hardware or frameworks. Instead, we implement a specific SNN architecture (CoLaNET) in Rust and optimize it for common computing platforms. As a case study, we demonstrate our implementation, called Spiffy, on a Raspberry Pi using the MNIST dataset. Spiffy achieves 92% accuracy with low latency - just 0.9 ms per training step and 0.45 ms per inference step. The code is open-source.
- Asia > Russia (0.05)
- Europe > Russia > Volga Federal District > Chuvash Republic > Cheboksary (0.05)
- Europe > Russia > Central Federal District > Moscow Oblast > Moscow (0.04)