The illusion of learning

The content of this post is mostly based on the paper Every Model Learned by Gradient Descent Is Approximately a Kernel Machine by Pedro Domingos (november 2020). By examining how they work, deep neural networks convey a vague idea of "learning": an input is fed into the network, the machine transforms the input and the result is compared with the real observation, then the network is updated to enhance its performance; repeat this many times and the network will "learn" to do well. It seems like a cognitive "dynamical" process. Another common belief is that deep networks have the ability to automatically discover new representations of the data. The so called memory-based algorithms give, instead, a vague idea of staticity, firmness, cataloging and comparison.