CS 229 - Deep Learning Cheatsheet

Neural networks are a class of models that are built with layers. Commonly used types of neural networks include convolutional and recurrent neural networks. Activation function Activation functions are used at the end of a hidden unit to introduce non-linear complexities to the model. Learning rate The learning rate, often noted $\alpha$ or sometimes $\eta$, indicates at which pace the weights get updated. This can be fixed or adaptively changed.