This work analyses the text memorization behavior of large language models (LLMs) when subjected to nucleus sampling. Stochastic decoding methods like nucleus sampling are typically applied to overcome issues such as monotonous and repetitive text generation, which are often observed with maximization-based decoding techniques. We hypothesize that nucleus sampling might also reduce the occurrence of memorization patterns, because it could lead to the selection of tokens outside the memorized sequence. To test this hypothesis we create a diagnostic dataset with a known distribution of duplicates that gives us some control over the likelihood of memorization of certain parts of the training data. Our analysis of two GPT-Neo models fine-tuned on this dataset interestingly shows that (i) an increase of the nucleus size reduces memorization only modestly, and (ii) even when models do not engage in "hard" memorization -- a verbatim reproduction of training samples -- they may still display "soft" memorization whereby they generate outputs that echo the training data but without a complete one-by-one resemblance.

I often get pitched with a superior deep learning solution for Natural Language Understanding (NLU). After all, deep learning is the disruptive new force in AI. A better NLU AI entices many useful advancements, ranging from smarter chat bots and virtual assistants to news categorization, with an ultimate promise of better language comprehension. Lets assume this superior deep learning (DL) "product" is called "(dot)AI". Their pitch deck will invariably have a bar chart that looks something like this -- the claim being that the new DL topic classifier/tagger of (Dot)AI is better than state of the art methods.

In 1960, the physicist Eugene Wigner wrote a famous essay titled "The Unreasonable Effectiveness of Mathematics in the Natural Sciences" in which he explored the question of why mathematics is so remarkably useful in the natural sciences. A contemporary example of such "unreasonable effectiveness" is the success that machine learning has had in transforming many disciplines in the past decade. Particularly impressive is the progress in autonomous vehicles. In the 2004 DARPA Grand Challenge for autonomous vehicles, which popularized the idea of driverless cars, none of the vehicles was able to complete a relatively simple route through the Mojave Desert, and I thought it unlikely that I would see driverless cars operating in urban environments in my lifetime. Since that time, progress in this area has been phenomenal, thanks to rapid advances in using machine learning for sensing and navigation (and in building low-cost sensors and controls).