In this post we will see how to vectorize across multiple training examples. The outcome will be similar to what we saw in Logistic Regression. These equations tell us how, when given an input feature vector \(x \), we can generate predictions. If we have \(m \) training examples we need to repeat this proces \(m \) times. The notation \( a {[2](i)} \) means that we are talking about activation in the second layer that comes from \(i {th} \) training example.

When we are programming Logistic Regression or Neural Networks we should avoid explicit \(for \) loops. It's not always possible, but when we can, we should use built-in functions or find some other ways to compute it. Vectorizing the implementation of Logistic Regression makes the code highly efficient. In this post we will see how we can use this technique to compute gradient descent without using even a single \(for \) loop. This code was non-vectorized and highly inefficent so we need to transform it.

When we are programming Logistic Regression or Neural Networks we should avoid explicit \(for \) loops. It's not always possible, but when we can, we should use built-in functions or find some other ways to compute it. Vectorizing the implementation of Logistic Regression makes the code highly efficient. In this post we will see how we can use this technique to compute gradient descent without using even a single \(for \) loop. This code was non-vectorized and highly inefficent so we need to transform it.

We're going to take this \(400 \) units and build the next layer with \(120 \) units. So, this is actually our first \(Fully \enspace connected \) layer. In this layer we have \(400 \) units densely connected to \(120 \) units. This \(Fully \enspace connected \) layer is like the single neural network layer. Hence, this is just a standard neural network where you have a weight matrix that's called \(W \) of a dimension \(120 \times 400 \).

Next we will apply another \(pooling \) layer with filter size \(f 2 \), and stride \(s 2\) so once again we reduce the size of an image by \(2 \) (as we did with the first \(pooling \) layer). Finally we have \(5\times5\times16\) volume and if we multiply these numbers \(5\times5\times16\) we get \(400\). We reduced dimensions of an image so now we can apply a \(Fully\enspace connected\) layer with \(120\) nodes. Then we apply another \(Fully\enspace connected\) layer with \(84 \) nodes. The final step is to use these \(84 \) features to get the final output, and at the output can take on \(10 \) possible values because we have to recognize \(10 \) different digits (\(0\) to\(9 \)), so at the end we have a \(softmax \) layer with a \(10 \)-way classification output (although back then \(LeNet-5 \) actually used a different classifier at the output layer, one that's useless today).

In the previous posts we talked about LeNet 5. Let's now see one more example of a convolutional neural network. The second convolutional neural network that we are going to present is \(AlexNet \) neural network. An input to this neural network is \(227\times227\times3\). We have a color image as an input and that is why we have \(3 \) channels. Let's explore the architecture of this convolutional neural network.

When we are programming Logistic Regression or Neural Networks we should avoid explicit \(for \) loops. It's not always possible, but when we can, we should use built-in functions or find some other ways to compute it. Vectorizing the implementation of Logistic Regression makes the code highly efficient. In this post we will see how we can use this technique to compute gradient descent without using even a single \(for \) loop. This code was non-vectorized and highly inefficent so we need to transform it.