Inspired by the workings of biological brains, humans have designed artificial neural networks (ANNs), sparking profound advancements across various fields. However, the biological brain possesses high plasticity, enabling it to develop simple, efficient, and powerful structures to cope with complex external environments. In contrast, the superior performance of ANNs often relies on meticulously crafted architectures, which can make them vulnerable when handling complex inputs. Moreover, overparameterization often characterizes the most advanced ANNs. This paper explores the path toward building streamlined and plastic ANNs.

All my Gen Z friends want to ditch their smartphones. But there's more at stake than they think. My friend Lilah is the crunchiest person I know. She refuses to kill bugs and rats. She once made me try her homemade wine (disastrous). A few years ago, she quit her food-justice nonprofit job to live in a yurt, and after that she went to grad school and moved into an attic, where her roommates were squirrels. Against her will, she did own an iPhone for a time.

This paper asks a set of well-motivated questions about learning of sensory representations by biological brains through experience, and proposes a continuous two-dimensional space of candidate learning rules, parameterized by levels of top-down feedback and Hebbian learning. They first show that this space contains five important candidate learning algorithms as specific points, such as Gradient Descent and Contrastive Hebbian. They then analyze the learning dynamics of these rules in a linear network with one hidden layer, trained to learn a hierarchy of concepts, following the previous work, and identify zones where deep networks exhibit qualitative signatures of biological learning. The work includes an interesting way to parameterize learning rules and aims to tackles the well-motivated problem of characterizing which learning rule is implemented in the biological brain. The model used in the paper is overly simple.

Neuron importance assessment is crucial for understanding the inner workings of artificial neural networks (ANNs) and improving their interpretability and efficiency. This paper introduces a novel approach to neuron significance assessment inspired by frequency tagging, a technique from neuroscience. By applying sinusoidal contrast modulation to image inputs and analyzing resulting neuron activations, this method enables fine-grained analysis of a network's decision-making processes. Experiments conducted with a convolutional neural network for image classification reveal notable harmonics and intermodulations in neuron-specific responses under part-based frequency tagging. These findings suggest that ANNs exhibit behavior akin to biological brains in tuning to flickering frequencies, thereby opening avenues for neuron/filter importance assessment through frequency tagging. The proposed method holds promise for applications in network pruning, and model interpretability, contributing to the advancement of explainable artificial intelligence and addressing the lack of transparency in neural networks. Future research directions include developing novel loss functions to encourage biologically plausible behavior in ANNs.

Achieving General Artificial Intelligence (AGI) has long been a grand challenge in the field of AI, and brain-inspired computing is widely acknowledged as one of the most promising approaches to realize this goal. This paper introduces a novel brain-inspired AI framework, Orangutan. It simulates the structure and computational mechanisms of biological brains on multiple scales, encompassing multi-compartment neuron architectures, diverse synaptic connection modalities, neural microcircuits, cortical columns, and brain regions, as well as biochemical processes including facilitation, feedforward inhibition, short-term potentiation, and short-term depression, all grounded in solid neuroscience. Building upon these highly integrated brain-like mechanisms, I have developed a sensorimotor model that simulates human saccadic eye movements during object observation. The model's algorithmic efficacy was validated through testing with the observation of handwritten digit images.

From'The Terminator' to'I, Robot', killer robots have been a staple feature in science fiction blockbusters for years. But nightmares of AI overtaking humanity might be further away than we thought, according to scientists. New research from the University of Oxford suggests that the human brain learns information in a fundamentally different and more efficient way to machines. This allows humans to learn something after seeing it once, while AI needs to be trained hundreds of times on the same information. Unlike AI, humans can also learn new information without it interfering with the knowledge we already have.

Top-down connections in the biological brain has been shown to be important in high cognitive functions. However, the function of this mechanism in machine learning has not been defined clearly. In this study, we propose to lay out a framework constituted by a bottom-up and a top-down network. Here, we use a Top-down Credit Assignment Network (TDCA-network) to replace the loss function and back propagation (BP) which serve as the feedback mechanism in traditional bottom-up network training paradigm. Our results show that the credit given by well-trained TDCA-network outperforms the gradient from backpropagation in classification task under different settings on multiple datasets. In addition, we successfully use a credit diffusing trick, which can keep training and testing performance remain unchanged, to reduce parameter complexity of the TDCA-network. More importantly, by comparing their trajectories in the parameter landscape, we find that TDCA-network directly achieved a global optimum, in contrast to that backpropagation only can gain a localized optimum. Thus, our results demonstrate that TDCA-network not only provide a biological plausible learning mechanism, but also has the potential to directly achieve global optimum, indicating that top-down credit assignment can substitute backpropagation, and provide a better learning framework for Deep Neural Networks.

According to Wikipedia, an ANN is based on a collection of connected units or nodes called artificial neurons, which loosely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can transmit a signal to other neurons. An artificial neuron that receives a signal then processes it and can signal neurons connected to it. In ANN implementations, the "signal" at a connection is a real number, and the output of each neuron is computed by some non-linear function of the sum of its inputs. The connections are called edges.

It's a good thing that the abstract and the concrete (or abstract objects in "mathematical space" and the "real world") are brought together in David Chalmers' account of Strong Artificial Intelligence (AI). Often it's almost (or literally) as if AI theorists believe that (as it were) disembodied computations can themselves bring about mind or even consciousness.

Google, Facebook & Amazon all use deep learning methods, but how does it work? In this video Computerphile explains how deep learning works. Research Fellow & Deep Learning Expert Brais Martinez explains. Deep learning (also known as deep structured learning) is part of a broader family of machine learning methods based on artificial neural networks with representation learning. Learning can be supervised, semi-supervised or unsupervised.