Deep Learning, Structure and Innate Priors

Earlier this month, I had the exciting opportunity to moderate a discussion between Professors Yann LeCun and Christopher Manning, titled "What innate priors should we build into the architecture of deep learning systems?" The event was a special installment of AI Salon, a discussion series held within the Stanford AI Lab that often features expert guests. This discussion topic – about the structural design decisions we build into our neural architectures, and how those correspond to certain assumptions and inductive biases – is an important one in AI right now. In fact, last year I highlighted "the return of linguistic structure" as one of the top four NLP Deep Learning research trends of 2017. On one side, Manning is a prominent advocate for incorporating more linguistic structure into deep learning systems.