These limitations lead to reduced robustness and generalization, making it difficult to meet the stringent demands of intelligent manufacturing for high accuracy, adaptability, and real-time responsiveness [8]. For these challenges, Multimodal Large Language Models (MLLMs) have recently attracted significant attention as a promising paradigm for industrial vision inspection [9]. By integrating heterogeneous modalities such as images and text, MLLMs exhibit enhanced semantic understanding and cross-modal reasoning capabilities [10], which are crucial for accurately interpreting intricate industrial scenes. This capability enables them to identify subtle defect patterns that would otherwise be overlooked by unimodal vision systems [11]. However, the adoption of MLLMs in real-world industrial environments is hindered by their massive parameter scale, high computational cost, and substantial memory footprint [12]. Deploying these models directly on resource-constrained edge devices often proves infeasible [13], while cloud-only processing introduces latency that disrupts the stringent real-time requirements of industrial pipelines [14]. To address these issues, we propose SAEC, a scene-aware enhanced edge-cloud collaborative industrial vision inspection framework with MLLM. The central idea of SAEC is to integrate scene-aware mechanisms with an edge-cloud collaborative architecture in order to balance accuracy and efficiency. By harmonizing multimodal reasoning with adaptive task scheduling across edge and cloud resources, SAEC achieves robust defect detection under complex scenarios, while maintaining scalability, resource efficiency, and practicality for deployment in real-world manufacturing systems.

3D Gaussian Splats (3DGSs) are 3D object models derived from multi-view images. Such "digital twins" are useful for simulations, virtual reality, marketing, robot policy fine-tuning, and part inspection. 3D object scanning usually requires multi-camera arrays, precise laser scanners, or robot wrist-mounted cameras, which have restricted workspaces. We propose Omni-Scan, a pipeline for producing high-quality 3D Gaussian Splat models using a bi-manual robot that grasps an object with one gripper and rotates the object with respect to a stationary camera. The object is then re-grasped by a second gripper to expose surfaces that were occluded by the first gripper. We present the Omni-Scan robot pipeline using DepthAny-thing, Segment Anything, as well as RAFT optical flow models to identify and isolate objects held by a robot gripper while removing the gripper and the background. We then modify the 3DGS training pipeline to support concatenated datasets with gripper occlusion, producing an omni-directional (360 degree view) model of the object. We apply Omni-Scan to part defect inspection, finding that it can identify visual or geometric defects in 12 different industrial and household objects with an average accuracy of 83%. Interactive videos of Omni-Scan 3DGS models can be found at https://berkeleyautomation.github.io/omni-scan/

A Digital Twin (DT) replicates objects, processes, or systems for real-time monitoring, simulation, and predictive maintenance. Recent advancements like Large Language Models (LLMs) have revolutionized traditional AI systems and offer immense potential when combined with DT in industrial applications such as railway defect inspection. Traditionally, this inspection requires extensive defect samples to identify patterns, but limited samples can lead to overfitting and poor performance on unseen defects. Integrating pre-trained LLMs into DT addresses this challenge by reducing the need for vast sample data. We introduce DefectTwin, which employs a multimodal and multi-model (M^2) LLM-based AI pipeline to analyze both seen and unseen visual defects in railways. This application enables a railway agent to perform expert-level defect analysis using consumer electronics (e.g., tablets). A multimodal processor ensures responses are in a consumable format, while an instant user feedback mechanism (instaUF) enhances Quality-of-Experience (QoE). The proposed M^2 LLM outperforms existing models, achieving high precision (0.76-0.93) across multimodal inputs including text, images, and videos of pre-trained defects, and demonstrates superior zero-shot generalizability for unseen defects. We also evaluate the latency, token count, and usefulness of responses generated by DefectTwin on consumer devices. To our knowledge, DefectTwin is the first LLM-integrated DT designed for railway defect inspection.

Artificial Intelligence (AI)-driven defect inspection is pivotal in industrial manufacturing. Yet, many methods, tailored to specific pipelines, grapple with diverse product portfolios and evolving processes. Addressing this, we present the Incremental Unified Framework (IUF), which can reduce the feature conflict problem when continuously integrating new objects in the pipeline, making it advantageous in object-incremental learning scenarios. Employing a state-of-the-art transformer, we introduce Object-Aware Self-Attention (OASA) to delineate distinct semantic boundaries. Semantic Compression Loss (SCL) is integrated to optimize non-primary semantic space, enhancing network adaptability for novel objects. Additionally, we prioritize retaining the features of established objects during weight updates. Demonstrating prowess in both image and pixel-level defect inspection, our approach achieves state-of-the-art performance, proving indispensable for dynamic and scalable industrial inspections. Our code will be released at \url{https://github.com/jqtangust/IUF}.

The automatic inspection of surface defects is an important task for quality control in the computers, communications, and consumer electronics (3C) industry. Conventional devices for defect inspection (viz. line-scan sensors) have a limited field of view, thus, a robot-aided defect inspection system needs to scan the object from multiple viewpoints. Optimally selecting the robot's viewpoints and planning a path is regarded as coverage path planning (CPP), a problem that enables inspecting the object's complete surface while reducing the scanning time and avoiding misdetection of defects. However, the development of CPP strategies for robotic line scanners has not been sufficiently studied by researchers. To fill this gap in the literature, in this paper, we present a new approach for robotic line scanners to detect surface defects of 3C free-form objects automatically. Our proposed solution consists of generating a local path by a new hybrid region segmentation method and an adaptive planning algorithm to ensure the coverage of the complete object surface. An optimization method for the global path sequence is developed to maximize the scanning efficiency. To verify our proposed methodology, we conduct detailed simulation-based and experimental studies on various free-form workpieces, and compare its performance with a state-of-the-art solution. The reported results demonstrate the feasibility and effectiveness of our approach.

In many industries, advocates of artificial intelligence and autonomous technology are quick to promise sweeping transformation and fully autonomous solutions. However, the optimists usually promise more than they can deliver and soon find the engineering challenges are greater than they first realised. In this article, Zohar Kantor, vice president of sales at artificial intelligence start-up Lean.AI, asks whether we celebrated the arrival of autonomous quality inspection too soon. In 2013, Elon Musk said, "it's a bridge too far to go to fully autonomous cars." Although the world of driverless vehicles has moved on significantly since this admission, it was a belated recognition that the Tesla CEO had under-estimated the challenges of operating a vehicle without a human being in the driver's seat.

The total installed base of AI-enabled devices in industrial manufacturing is forecast to reach 15.4 million within five years, with a CAGR of 64.8% from 2019 to 2024. It's claimed that Artificial Intelligence (AI) will revolutionise the industrial manufacturing space, and in many respects that transformation has already begun. AI is already delivering generative design in product development, production forecasting in inventory management, and machine vision, defect inspection, production optimisation, and predictive maintenance in the production phase. "AI in industrial manufacturing is a story of edge implementation," says Lian Jye Su, principal analyst at global tech market advisory firm, ABI Research. "Since manufacturers are not comfortable having their data transferred to a public cloud, nearly all industrial AI training and inference workloads happen at the edge, namely on device, gateways and on-premise servers."