Overview
Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations
Schemmer, Max, Kühl, Niklas, Benz, Carina, Bartos, Andrea, Satzger, Gerhard
AI advice is becoming increasingly popular, e.g., in investment and medical treatment decisions. As this advice is typically imperfect, decision-makers have to exert discretion as to whether actually follow that advice: they have to "appropriately" rely on correct and turn down incorrect advice. However, current research on appropriate reliance still lacks a common definition as well as an operational measurement concept. Additionally, no in-depth behavioral experiments have been conducted that help understand the factors influencing this behavior. In this paper, we propose Appropriateness of Reliance (AoR) as an underlying, quantifiable two-dimensional measurement concept. We develop a research model that analyzes the effect of providing explanations for AI advice. In an experiment with 200 participants, we demonstrate how these explanations influence the AoR, and, thus, the effectiveness of AI advice. Our work contributes fundamental concepts for the analysis of reliance behavior and the purposeful design of AI advisors.
MLOps Spanning Whole Machine Learning Life Cycle: A Survey
Zhengxin, Fang, Yi, Yuan, Jingyu, Zhang, Yue, Liu, Yuechen, Mu, Qinghua, Lu, Xiwei, Xu, Jeff, Wang, Chen, Wang, Shuai, Zhang, Shiping, Chen
Google AlphaGos win has significantly motivated and sped up machine learning (ML) research and development, which led to tremendous ML technical advances and wider adoptions in various domains (e.g., Finance, Health, Defense, and Education). These advances have resulted in numerous new concepts and technologies, which are too many for people to catch up to and even make them confused, especially for newcomers to the ML area. This paper is aimed to present a clear picture of the state-of-the-art of the existing ML technologies with a comprehensive survey. We lay out this survey by viewing ML as a MLOps (ML Operations) process, where the key concepts and activities are collected and elaborated with representative works and surveys. We hope that this paper can serve as a quick reference manual (a survey of surveys) for newcomers (e.g., researchers, practitioners) of ML to get an overview of the MLOps process, as well as a good understanding of the key technologies used in each step of the ML process, and know where to find more details.
Bottleneck Analysis of Dynamic Graph Neural Network Inference on CPU and GPU
Chen, Hanqiu, Alhinai, Yahya, Jiang, Yihan, Na, Eunjee, Hao, Cong
Dynamic graph neural network (DGNN) is becoming increasingly popular because of its widespread use in capturing dynamic features in the real world. A variety of dynamic graph neural networks designed from algorithmic perspectives have succeeded in incorporating temporal information into graph processing. Despite the promising algorithmic performance, deploying DGNNs on hardware presents additional challenges due to the model complexity, diversity, and the nature of the time dependency. Meanwhile, the differences between DGNNs and static graph neural networks make hardware-related optimizations for static graph neural networks unsuitable for DGNNs. In this paper, we select eight prevailing DGNNs with different characteristics and profile them on both CPU and GPU. The profiling results are summarized and analyzed, providing in-depth insights into the bottlenecks of DGNNs on hardware and identifying potential optimization opportunities for future DGNN acceleration. Followed by a comprehensive survey, we provide a detailed analysis of DGNN performance bottlenecks on hardware, including temporal data dependency, workload imbalance, data movement, and GPU warm-up. We suggest several optimizations from both software and hardware perspectives. This paper is the first to provide an in-depth analysis of the hardware performance of DGNN Code is available at https://github.com/sharc-lab/DGNN_analysis.
A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services
Katare, Dewant, Perino, Diego, Nurmi, Jari, Warnier, Martijn, Janssen, Marijn, Ding, Aaron Yi
Autonomous driving services rely heavily on sensors such as cameras, LiDAR, radar, and communication modules. A common practice of processing the sensed data is using a high-performance computing unit placed inside the vehicle, which deploys AI models and algorithms to act as the brain or administrator of the vehicle. The vehicular data generated from average hours of driving can be up to 20 Terabytes depending on the data rate and specification of the sensors. Given the scale and fast growth of services for autonomous driving, it is essential to improve the overall energy and environmental efficiency, especially in the trend towards vehicular electrification (e.g., battery-powered). Although the areas have seen significant advancements in sensor technologies, wireless communications, computing and AI/ML algorithms, the challenge still exists in how to apply and integrate those technology innovations to achieve energy efficiency. This survey reviews and compares the connected vehicular applications, vehicular communications, approximation and Edge AI techniques. The focus is on energy efficiency by covering newly proposed approximation and enabling frameworks. To the best of our knowledge, this survey is the first to review the latest approximate Edge AI frameworks and publicly available datasets in energy-efficient autonomous driving. The insights and vision from this survey can be beneficial for the collaborative driving service development on low-power and memory-constrained systems and also for the energy optimization of autonomous vehicles.
Systemic Fairness
Ray, Arindam, Padmanabhan, Balaji, Bouayad, Lina
Machine learning algorithms are increasingly used to make or support decisions in a wide range of settings. With such expansive use there is also growing concern about the fairness of such methods. Prior literature on algorithmic fairness has extensively addressed risks and in many cases presented approaches to manage some of them. However, most studies have focused on fairness issues that arise from actions taken by a (single) focal decision-maker or agent. In contrast, most real-world systems have many agents that work collectively as part of a larger ecosystem. For example, in a lending scenario, there are multiple lenders who evaluate loans for applicants, along with policymakers and other institutions whose decisions also affect outcomes. Thus, the broader impact of any lending decision of a single decision maker will likely depend on the actions of multiple different agents in the ecosystem. This paper develops formalisms for firm versus systemic fairness, and calls for a greater focus in the algorithmic fairness literature on ecosystem-wide fairness - or more simply systemic fairness - in real-world contexts.
Quantifying and Explaining Machine Learning Uncertainty in Predictive Process Monitoring: An Operations Research Perspective
Mehdiyev, Nijat, Majlatow, Maxim, Fettke, Peter
In today's highly competitive and complex business environment, organizations are under constant pressure to optimize their performance and decision-making processes. According to Herbert Simon, enhancing organizational performance relies on effectively channeling finite human attention towards critical data for decision-making, necessitating the integration of information systems (IS), artificial intelligence (AI) and operations research (OR) insights [1]. Recent OR research provides evidence in support of this proposition, as the discipline has witnessed a transformation due to the abundant availability of rich and voluminous data from various sources coupled with advances in machine learning [2]. As of late, heightened academic attention has been devoted to prescriptive analytics, a discipline that suggests combining the results of predictive analytics with optimization techniques in a probabilistic framework to generate responsive, automated, restricted, time-sensitive, and ideal decisions [3]. The confluence of AI and OR is evident due to their interdependent and complementary nature, as both disciplines strive to augment decision-making processes through computational and mathematical methodologies [4].
Rule-based detection of access to education and training in Germany
Dörpinghaus, Jens, Samray, David, Helmrich, Robert
As a result of transformation processes, the German labor market is highly dependent on vocational training, retraining and continuing education. To match training seekers and offers, we present a novel approach towards the automated detection of access to education and training in German training offers and advertisements. We will in particular focus on (a) general school and education degrees and schoolleaving certificates, (b) professional experience, (c) a previous apprenticeship and (d) a list of skills provided by the German Federal Employment Agency. This novel approach combines several methods: First, we provide a mapping of synonyms in education combining different qualifications and adding deprecated terms. Second, we provide a rule-based matching to identify the need for professional experience or apprenticeship. However, not all access requirements can be matched due to incompatible data schemata or non-standardizes requirements, e.g initial tests or interviews. While we can identify several shortcomings, the presented approach offers promising results for two data sets: training and re-training advertisements.
Nextech3D.ai: Leading the Way in AI-Driven 3D Modeling for Ecommerce
With its breakthrough generative AI technology, Nextech3D.ai is poised to revolutionize 3D modeling applications, particularly in the fast-growing e-commerce industry, and emerge as a leader. The recent advent of ChatGPT, a sophisticated chatbot and trained language model, revolutionized the world of AI, bringing its vast potential and attention to the collective forefront of users and investors. AI-powered product offerings explicitly focused on 3D modeling for e-commerce serve a massive Total Addressable Market (TAM) and Serviceable Addressable Market (SAM). The estimated market size of the 3D modeling for e-commerce space is around $100 billion within the $5.5 trillion global e-commerce industry. With its suite of innovative products, Nextech3D.ai is already a preferred 3D model supplier for the e-commerce behemoth Amazon's private label products.
Key Trends in Generative AI. Generative AI has continued to grow…
Generative AI has continued to grow rapidly in 2023, with increasing interest from organizations and individuals looking to create realistic and personalized content using artificial intelligence. However, there are several challenges facing the widespread adoption of generative AI, including the difficulty of sharing custom retraining models, the complexity of running open-source models, and the lack of mechanisms to incentivize model creators. In this report, we will provide an overview of the current state of generative AI, its key trends, challenges, and opportunities. Generative AI refers to the use of artificial intelligence to generate new content, such as images, text, or music. Generative AI has the potential to revolutionize the creative industries, enabling organizations and individuals to produce high-quality, personalized content at scale. However, the development and deployment of generative AI models pose several challenges, including technical, legal, and ethical issues.
A Survey of Resources and Methods for Natural Language Processing of Serbian Language
The Serbian language is a Slavic language spoken by over 12 million speakers and well understood by over 15 million people. In the area of natural language processing, it can be considered a low-resourced language. Also, Serbian is considered a high-inflectional language. The combination of many word inflections and low availability of language resources makes natural language processing of Serbian challenging. Nevertheless, over the past three decades, there have been a number of initiatives to develop resources and methods for natural language processing of Serbian, ranging from developing a corpus of free text from books and the internet, annotated corpora for classification and named entity recognition tasks to various methods and models performing these tasks.