On the Necessity of Metalearning: Learning Suitable Parameterizations for Learning Processes

In this paper we will discuss metalearning and how we can go beyond the current classical learning paradigm. We will first address the importance of inductive biases in the learning process and what is at stake: the quantities of data necessary to learn. We will subsequently see the importance of choosing suitable parameterizations to end up with well-defined learning processes. Especially since in the context of real-world applications, we face numerous biases due, e.g., to the specificities of sensors, the heterogeneity of data sources, the multiplicity of points of view, etc. This will lead us to the idea of exploiting the structuring of the concepts to be learned in order to organize the learning process that we published previously. We conclude by discussing the perspectives around parameter-tying schemes and the emergence of universal aspects in the models thus learned. Metalearning (learning-to-learn) offers promising levels of flexibility and generalization while reducing the quantities of data needed to learn (or adapt). Few-shot and zero-shot learning are examples of metalearning approaches that allow easy adaptation to new tasks (or domains), using few examples for the former or no examples at all for the latter. Metalearning involves the study of regularities (structural dependencies) across models and tasks, where "task" is taken in its broader sense and includes the classical learning tasks, e.g., image classification and segmentation, activity recognition from on-body sensor deployments, etc.; robot configurations, e.g., Cully et al. (2015); topologies of sensor deployment, e.g., Hamidi & Osmani (2021); multiple views (or perspectives) on a given phenomena, e.g., Hamidi et al. (2020); clients in a federated deployment, e.g., Hamidi & Osmani (2022); What characterizes a task is the tailored family of inductive biases (search or representation) that makes the learning process converge into a satisfactory solution.