Towards certification: A complete statistical validation pipeline for supervised learning in industry

The field of Machine Learning (ML) [1, 2] and its broad spectrum of applications has revolutionized a plethora of technological industries in recent years ranging from the energy sector or material sciences to telecommunications, finance or consumer goods, to cite some [3]. In the context of aeronautical engineering and aerospace technologies, the field has embraced ML tools only in recent years, and impact is growing at a rapid pace, ranging from generalpurpose ML-based fluid mechanics [4-6], aeroacoustics [7], wind turbines [8] or aerostructures [9] (including prediction of landing gear loads [10]) to flight trajectories optimization [11] or enhancing predictive maintenance [12, 13]: see the recent and illuminating reviews [14, 15] and references therein. Interestingly, the integration of ML-related tools and ideas in the aeronautical and aerospace industries is still in its infancy. Part of the reason is that any new technology has a necessary adoption curve [16, 17], and the fact that ML-solutions require expert knowledge at the crossroads of computer science and statistics -and a sophisticated operationalization infrastructure (MLOps) [18] - does not facilitate this adoption. However, a deeper reason is probably impeding faster adoption: while ML-technologies promise high performance and reduction in development and operating costs [19] (e.g. by reducing costs related to expensive and lengthy wind tunnel experiments and numerical simulations), ensuring adequate safety remains paramount in aeronautical industries, and ML-based tools are often seen as sophisticated black-boxes that suffer from low degree of trustability, and thus difficult to validate their safety. Therefore, air safety authorities demand rigorous validation and verification processes for these models, and industry leaders have started to propose guidelines and a roadmap on concepts of design assurance for neural network-related technologies [20-22]. However, only very recently industry has started to embrace the complexities of certifying ML models [23-27], prompting the initiation of discussions around the development of guidelines and a roadmap for design assurance, especially concerning network-related technologies. This pressing need underscores the imperative for collaborative efforts within the industry to establish robust validation frameworks that not only meet regulatory standards but also address the evolving challenges posed by ML integration. This has indeed been well understood and undertaken by Airbus who has established an internal working group on verification and validation of surrogate models in the frame of loads and stress domains.