We could not have written this Best of AI without mentioning the 7th edition of the International Conference on Learning Representation (ICLR) which took place in New Orleans at the beginning of the month. Did not get the chance to go to this top-notch research event? We came across a summary of the conference by a researcher from NVIDIA labs. This technical article will give you a great overview of some of the hottest keywords in AI research. Look them up if you do not get them all!

Abstract: Natural language is hierarchically structured: smaller units (e.g., phrases) are nested within larger units (e.g., clauses). When a larger constituent ends, all of the smaller constituents that are nested within it must also be closed. While the standard LSTM architecture allows different neurons to track information at different time scales, it does not have an explicit bias towards modeling a hierarchy of constituents. This paper proposes to add such an inductive bias by ordering the neurons; a vector of master input and forget gates ensures that when a given neuron is updated, all the neurons that follow it in the ordering are also updated. Our novel recurrent architecture, ordered neurons LSTM (ON-LSTM), achieves good performance on four different tasks: language modeling, unsupervised parsing, targeted syntactic evaluation, and logical inference.