Playing Mortal Kombat with TensorFlow.js. Transfer learning and data augmentation

I started looking at deep learning. I've focused mainly on recurrent neural networks (RNNs), specifically LSTM because of their "unreasonable effectiveness" in the domain of Guess.js. In the same time, I started playing with convolutional neural networks (CNNs), which although less traditionally, are also often used for time series. CNNs are usually used for image classification, recognition, and detection. After playing around with CNNs, I remembered an experiment I did a few years ago, when the browser vendors introduced the getUserMedia API. In this experiment, I used the user's camera as a controller for playing a small JavaScript clone of Mortal Kombat 3. You can find the game at my GitHub account. The algorithm is so simple that I'd be able to explain it in a few sentences: The algorithm takes a snapshot of the background behind the user.