Theophano Mitsa, Ph.D. on LinkedIn: Another great post from Damien!

I once had an interview where I was asked to design a model that would recommend ads to users. I simply drew on the board a simple Recommender engine with a User embedding and an Ads embedding, a couple of non-linear interactions and a "click-or-not" learning task. But the interviewer asked "but wait, we have billions of users, how is this model going to fit on a server?!". That was a great question, and I failed the interview! A naive embedding encoding strategy will assign a vector to each of the categories seen during training, and an "unknown" vector for all the categories seen during serving but not at training time. That can be a relatively safe strategy for NLP problems if you have a large enough training set as the set of possible words or tokens can be quite static.