Advances in Machine Learning for the Behavioral Sciences

This is most apparent when auto-encoders are trained, where a network is trained to map the input data upon itself but is forced to project them into a lower-dimensional embedding space on the way (Vincent et al., 2010). In addition to the conventional fully connected layers, there are various special types of network connections. For example, in computer vision, convolu-tional layers are commonly used, which train multiple sliding windows that move over the image data and process just a part of the image at a time, thereby learning to recognize local features. These layers are subsequently abstracted into more and more complex visual patterns (Krizhevsky et al., 2017). For temporal data, one can use recurrent neural networks, which do not make predictions for individual input vectors, but for a sequence of input vectors. To do so, they allow feeding abstracted information from previous data points forward to the next layers.