Whether deep neural networks can exhibit emergent behaviour is not only relevant for understanding how deep learning works, it is also pivotal for estimating potential security risks of increasingly capable artificial intelligence systems. Here, we show that training deep neural networks gives rise to emergent weight morphologies independent of the training data. Specifically, in analogy to condensed matter physics, we derive a theory that predict that the homogeneous state of deep neural networks is unstable in a way that leads to the emergence of periodic channel structures. We verified these structures by performing numerical experiments on a variety of data sets. Our work demonstrates emergence in the training of deep neural networks, which impacts the achievable performance of deep neural networks.

Explainability is becoming an increasingly important topic for deep neural networks. Though the operation in convolutional layers is easier to understand, processing becomes opaque in fully-connected layers. The basic idea in our work is that each instance, as it flows through the layers, causes a different activation pattern in the hidden layers and in our Paths methodology, we cluster these activation vectors for each hidden layer and then see how the clusters in successive layers connect to one another as activation flows from the input layer to the output. We find that instances of the same class follow a small number of cluster sequences over the layers, which we name ``decision paths." Such paths explain how classification decisions are typically made, and also help us determine outliers that follow unusual paths. We also propose using the Sankey diagram to visualize such pathways. We validate our method with experiments on two feed-forward networks trained on MNIST and CELEB data sets, and one recurrent network trained on PenDigits.

Deep Learning is a subfield of machine learning: a new take on learning representations from data that puts emphasis on learning successive layers of increasingly meaningful representations. How many layers contribute to a model of the data is called the depth of the model. The specification of what a layer does to its input data is stored in the layer's weights, which in essence are a bunch of numbers. In technical terms, we'd say that the transformation implemented by a layer is parameterized by its weights. Wights are also called the parameters of a layer.

In this post we covered a brief history of AI and how it evolved over the years through symbolic AI, ML and DL. We also tried to understand how AI, ML and DL are related/correlated with one other (figure 6). These terms often muddle a lot of people and are often used interchangeably. There is a lot of hype around DL models due to their ability to better deal with complex problems such as image classification, text processing etc. Additionally these models also remove the necessity to manually engineer features, amenable to the objective at hand. Therefore, in the next post we shall be delving into the details of DL models and try to understand its various components along with examples.

Deep Learning, many of us using this tool to solve the most complicated problems even without understanding it. Tensorflow framework and Keras Interface makes life easy for everyone. But it is necessary to know the basic concepts of Neural Networks to make your solutions more precise. Let me share my understandings. Imagine that we have a problem, the bank gave all of the transaction details such as customer name, age, bank balance, retired or not, transactions and so on.

It's common knowledge that neural networks are really good at one narrow task, but they fail at handling multiple tasks. This is unlike the human brain which is able to use the same concepts at amazingly diverse tasks. For example, if you have never seen a fractal before and I show you one right now. After seeing the image of a fractal, you'll be able to handle multiple tasks related to it: How are you able to do all these tasks? Are there dedicated neural networks in your brain specializing in all these tasks?

A concise definition of the field would be as follows: the effort to automate intellectual tasks normally performed by humans. Artificial intelligence was born in the 1950s, when a handful of pioneers from the nascent field of computer science started asking whether computers could be made to "think". As such, AI is a general field that encompasses machine learning and deep learning, but that also includes many more approaches that don't involve any learning. Many experts believed that human-level artificial intelligence could be achieved by having programmers handcraft a sufficiently large set of explicit rules for manipulating knowledge. This approach is known as symbolic AI,A new approach arose to take symbolic AI's place: machine learning.

A concise definition of the field would be as follows: the effort to automate intellectual tasks normally performed by humans. Artificial intelligence was born in the 1950s, when a handful of pioneers from the nascent field of computer science started asking whether computers could be made to "think". As such, AI is a general field that encompasses machine learning and deep learning, but that also includes many more approaches that don't involve any learning. Many experts believed that human-level artificial intelligence could be achieved by having programmers handcraft a sufficiently large set of explicit rules for manipulating knowledge. This approach is known as symbolic AI,A new approach arose to take symbolic AI's place: machine learning.

The Human's are evolved about 2.3 to 2.4 million years ago. Since the 18th century, Scientists thought the great apes to be closely related to human beings. In the 19th century, They speculated that closest living relatives of humans were either chimpanzees or gorillas. Do you know what made us different from our closest living relatives? Humans have a persistent process of thinking.

As deep neural networks grow in size, from thousands to millions to billions of weights, the performance of those networks becomes limited by our ability to accurately train them. A common naive question arises: if we have a system with billions of degrees of freedom, don't we also need billions of samples to train it? Of course, the success of deep learning indicates that reliable models can be learned with reasonable amounts of data. Similar questions arise in protein folding, spin glasses and biological neural networks. With effectively infinite potential folding/spin/wiring configurations, how does the system find the precise arrangement that leads to useful and robust results? Simple sampling of the possible configurations until an optimal one is reached is not a viable option even if one waited for the age of the universe. On the contrary, there appears to be a mechanism in the above phenomena that forces them to achieve configurations that live on a low-dimensional manifold, avoiding the curse of dimensionality. In the current work we use the concept of mutual information between successive layers of a deep neural network to elucidate this mechanism and suggest possible ways of exploiting it to accelerate training. We show that adding structure to the neural network that enforces higher mutual information between layers speeds training and leads to more accurate results. High mutual information between layers implies that the effective number of free parameters is exponentially smaller than the raw number of tunable weights.