[D] Simple Questions Thread August 02, 2020

I'm new to deep learning and started by implementing an autoencoder for time-series data, which seemed simple enough, or so I thought. However, the model performance gets worse (even on training data) as I make the model deeper, which doesn't make any sense to me. Here's my first autoencoder (model 1), implemented in PyTorch: I am able to train this model with a training set of over 200k examples using MSELoss() and Adam optimizer (LR 1e-3). But the loss (even for the training set) doesn't go down as low as I want it to go, so I made the model just one layer deeper at each stage (model 2) to see if it would train better: As you can see, all I am doing is adding an extra fully-connected layer to each stage of the auto-encoder. This is a very simple change that I thought would improve performance.