Towards Improved Research Methodologies for Industrial AI: A case study of false call reduction

--Are current artificial intelligence (AI) research methodologies ready to create successful, productive, and profitable AI applications? This work presents a case study on an industrial AI use case called false call reduction for automated optical inspection to demonstrate the shortcomings of current best practices. We identify seven weaknesses prevalent in related peer-reviewed work and experimentally show their consequences. We show that the best-practice methodology would fail for this use case. We argue amongst others for the necessity of requirement-aware metrics to ensure achieving business objectives, clear definitions of success criteria, and a thorough analysis of temporal dynamics in experimental datasets. Our work encourages researchers to critically assess their methodologies for more successful applied AI research. The rise of automation in manufacturing has brought significant advancements to production processes. However, are current artificial intelligence (AI) research methodologies ready to create successful, productive, and profitable AI applications? Despite extensive research, the success of industrial AI applications has not kept pace with other industrial automation technologies due to methodological weaknesses. In this work, we address these methodological flaws using a case study on false call reduction in automated optical inspection (AOI) of printed circuit boards (PCBs). AOI systems, which use computer vision to inspect soldering quality, often produce a high number of false calls--incorrect classifications of non-defective PCBs as defective. These false calls consume valuable human resources in manual inspection stages. Our study identifies seven prevalent weaknesses in related research on this topic and demonstrates their negative impacts experimentally.