One of the more painful things about training Deep Neural Networks is the large number of hyperparameters you have to deal with. These could be your learning rate α, the discounting factor ρ, and epsilon ε if you are using the RMSprop optimizer (Hinton et al.) or the exponential decay rates β₁ and β₂ if you are using the Adam optimizer (Kingma et al.). You also need to choose the number of layers in the network or the number of hidden units for the layers. You might be using learning rate schedulers and would want to configure those features and a lot more! We definitely need ways to better organize our hyperparameter tuning process.

Our control model was a well established machine learning model using features that are known to work well. For text, the features are essentially normalized word counts (TF-IDF: term frequency / inverse document frequency vectors). For images, we use HOG features (histogram of oriented gradients). These features were fed into a logistic regression model for training and prediction. Our test model used custom collection; we fed data, trained a model, and made a prediction using transfer learning for text and image analysis under the covers.