In this video we build on last week Multilayer perceptrons to allow for more flexibility in the architecture! However, we need to be careful about the layer of abstraction we put in place in order to facilitate the work of the user who want to simply fit and predict. Here we make use of the following three concept: Network, Layer and Neuron. These three components will be composed together to make a fully connected feedforward neural network neural network. For those who don't know a fully connected feedforward neural network is defined as follows (From Wikipedia): "A feedforward neural network is an artificial neural network wherein connections between the nodes do not form a cycle. As such, it is different from its descendant: recurrent neural networks. The feedforward neural network was the first and simplest type of artificial neural network devised. In this network, the information moves in only one direction, forward, from the input nodes, through the hidden nodes (if any) and to the output nodes. There are no cycles or loops in the network."
On-chip edge intelligence has necessitated the exploration of algorithmic techniques to reduce the compute requirements of current machine learning frameworks. This work aims to bridge the recent algorithmic progress in training Binary Neural Networks and Spiking Neural Networks--both of which are driven by the same motivation and yet synergies between the two have not been fully explored. We show that training Spiking Neural Networks in the extreme quantization regime results in near full precision accuracies on large-scale datasets like CIFAR-100 and ImageNet. An important implication of this work is that Binary Spiking Neural Networks can be enabled by "In-Memory" hardware accelerators catered for Binary Neural Networks without suffering any accuracy degradation due to binarization. We utilize standard training techniques for non-spiking networks to generate our spiking networks by conversion process and also perform an extensive empirical analysis and explore simple design-time and run-time optimization techniques for reducing inference latency of spiking networks (both for binary and full-precision models) by an order of magnitude over prior work.
In the field of Artificial Intelligence, we tend to move full speed ahead in building and training models without considering the philosophical aspects of what we're doing. It's great to focus on development, but we also have a responsibility to consider the bigger picture from time to time. One of the things that drew me to Artificial Intelligence in the first place is that there is an equal amount of philosophical and technical depth to the discipline. Furthermore, I have found that the philosophy and the theory enrich one another. One question AI Engineers don't often stop to think about is, ironically, the concept of Intelligence itself.
Artificial Intelligence sounds freaking amazing: humanoid robots, artificial conscious, self learning systems and understanding the human brain. I won't lie; these were the things that motivated me to look into Artificial Intelligence. And till a certain extent they still do. I started out doing Physics and Life Sciences. One thing that caught my attention was the advancements in the field of so called "Artificial Neural Networks".