NIPS 2017 -- Day 1 Highlights – Insight Data

This talk gave a solid overview of the current state and recent advances in Deep Learning. Convolutional Neural Networks (CNN) and autoregressive models are starting to see ubiquitous use in production, showing a fast transition from research to industry. These models have taught us that introducing inductive biases such as translation invariance (CNN) or time recurrence (Recurrent Neural Networks) can be extremely useful. We've also found out that simple "tricks" such as Residual Networks or Attention can lead to tremendous leaps in performance. There are good reasons to believe we will find more such "tricks".