Wilson, Aaron
Explainable AI for Multivariate Time Series Pattern Exploration: Latent Space Visual Analytics with Temporal Fusion Transformer and Variational Autoencoders in Power Grid Event Diagnosis
Xu, Haowen, Boyaci, Ali, Lian, Jianming, Wilson, Aaron
Detecting and analyzing complex patterns in multivariate time-series data is crucial for decision-making in urban and environmental system operations. However, challenges arise from the high dimensionality, intricate complexity, and interconnected nature of complex patterns, which hinder the understanding of their underlying physical processes. Existing AI methods often face limitations in interpretability, computational efficiency, and scalability, reducing their applicability in real-world scenarios. This paper proposes a novel visual analytics framework that integrates two generative AI models, Temporal Fusion Transformer (TFT) and Variational Autoencoders (VAEs), to reduce complex patterns into lower-dimensional latent spaces and visualize them in 2D using dimensionality reduction techniques such as PCA, t-SNE, and UMAP with DBSCAN. These visualizations, presented through coordinated and interactive views and tailored glyphs, enable intuitive exploration of complex multivariate temporal patterns, identifying patterns' similarities and uncover their potential correlations for a better interpretability of the AI outputs. The framework is demonstrated through a case study on power grid signal data, where it identifies multi-label grid event signatures, including faults and anomalies with diverse root causes. Additionally, novel metrics and visualizations are introduced to validate the models and evaluate the performance, efficiency, and consistency of latent maps generated by TFT and VAE under different configurations. These analyses provide actionable insights for model parameter tuning and reliability improvements. Comparative results highlight that TFT achieves shorter run times and superior scalability to diverse time-series data shapes compared to VAE. This work advances fault diagnosis in multivariate time series, fostering explainable AI to support critical system operations.
Do Large Code Models Understand Programming Concepts? A Black-box Approach
Hooda, Ashish, Christodorescu, Mihai, Allamanis, Miltos, Wilson, Aaron, Fawaz, Kassem, Jha, Somesh
Large Language Models' success on text generation has also made them better at code generation and coding tasks. While a lot of work has demonstrated their remarkable performance on tasks such as code completion and editing, it is still unclear as to why. We help bridge this gap by exploring to what degree auto-regressive models understand the logical constructs of the underlying programs. We propose Counterfactual Analysis for Programming Concept Predicates (CACP) as a counterfactual testing framework to evaluate whether Large Code Models understand programming concepts. With only black-box access to the model, we use CACP to evaluate ten popular Large Code Models for four different programming concepts. Our findings suggest that current models lack understanding of concepts such as data flow and control flow.
Disentangling Space and Time in Video with Hierarchical Variational Auto-encoders
Grathwohl, Will, Wilson, Aaron
There are many forms of feature information present in video data. Principle among them are object identity information which is largely static across multiple video frames, and object pose and style information which continuously transforms from frame to frame. Most existing models confound these two types of representation by mapping them to a shared feature space. In this paper we propose a probabilistic approach for learning separable representations of object identity and pose information using unsupervised video data. Our approach leverages a deep generative model with a factored prior distribution that encodes properties of temporal invariances in the hidden feature set. Learning is achieved via variational inference. We present results of learning identity and pose information on a dataset of moving characters as well as a dataset of rotating 3D objects. Our experimental results demonstrate our model's success in factoring its representation, and demonstrate that the model achieves improved performance in transfer learning tasks.
Graph Analysis for Detecting Fraud, Waste, and Abuse in Healthcare Data
Liu, Juan (Medallia) | Bier, Eric (Palo Alto Research Center) | Wilson, Aaron (Palo Alto Research Center) | Guerra-Gomez, John Alexis (Yahoo Labs) | Honda, Tomonori (Inflection.com) | Sricharan, Kumar (Palo Alto Research Center) | Gilpin, Leilani (Massachusetts Institute for Technology) | Davies, Daniel (Palo Alto Research Center)
Detection of fraud, waste, and abuse (FWA) is an important yet challenging problem. In this article, we describe a system to detect suspicious activities in large healthcare datasets. Each healthcare dataset is viewed as a heterogeneous network consisting of millions of patients, hundreds of thousands of doctors, tens of thousands of pharmacies, and other entities. Graph analysis techniques are developed to find suspicious individuals, suspicious relationships between individuals, unusual changes over time, unusual geospatial dispersion, and anomalous network structure.
Graph Analysis for Detecting Fraud, Waste, and Abuse in Healthcare Data
Liu, Juan (Medallia) | Bier, Eric (Palo Alto Research Center) | Wilson, Aaron (Palo Alto Research Center) | Guerra-Gomez, John Alexis (Yahoo Labs) | Honda, Tomonori (Inflection.com) | Sricharan, Kumar (Palo Alto Research Center) | Gilpin, Leilani (Massachusetts Institute for Technology) | Davies, Daniel (Palo Alto Research Center)
Healthcare-related programs include federal and series of technical challenges. From a data representation state government programs such as Medicaid, view, healthcare data sets are often large and Medicare Advantage (Part C), Medicare FFS, and diverse. It is common to see a state's Medicaid program Medicare Prescription Drug Benefit (Part D). Nonhealth-care or a private healthcare insurance program having programs include Earned Income Tax hundreds of millions of claims per year, involving Credit (EITC), Pell Grants, Public Housing/Rental millions of patients and hundreds of thousands of Assistance, Retirement, Survivors and Disability Insurance providers of various types, for example, physicians, (RSDI), School Lunch, Supplemental Nutrition pharmacies, clinics and hospitals, and laboratories. Assistance Program (SNAP), Supplemental Security Any fraud-detection system needs to be able to handle Income (SSI), Unemployment Insurance (UI), and the large data volume and data diversity. While healthcare data (insurance claims, health Data patterns from both sides are dynamic. The complexity records, clinical data, provider information, and others) of the problem calls for a rich set of techniques offers tantalizing opportunities, it also poses a to examine healthcare data. Healthcare financials are complex, involving a from a suspicious individual or activity (as singled multitude of providers (physicians, pharmacies, clinics out by the automated screening components) and and hospitals, and laboratories), payers (insurance interacts with the system to navigate through data plans), and patients. To design a good fraud-detection items and collect evidence to build an investigation system, one must have a deep understanding of the case. The two categories have quite different technical financial incentives of all parties. Starting from database indexing/caching for fast data retrieval and domain knowledge, auditors and investigators have user interface design for intuitive user-system interaction.
Graph Analysis for Detecting Fraud,Waste, and Abuse in Healthcare Data
Liu, Juan (Palo Alto Research Center) | Bier, Eric (Palo Alto Research Center) | Wilson, Aaron (Palo Alto Research Center) | Honda, Tomo (Palo Alto Research Center) | Kumar, Sricharan (Palo Alto Research Center) | Gilpin, Leilani (Palo Alto Research Center) | Guerra-Gomez, John (Palo Alto Research Center) | Davies, Daniel (Palo Alto Research Center)
Detection of fraud, waste, and abuse (FWA) is an important yet difficult problem. In this paper, we describe a system to detect suspicious activities in large healthcare claims datasets. Each healthcare dataset is viewed as a heterogeneous network of patients, doctors, pharmacies, and other entities. These networks can be large, with millions of patients, hundreds of thousands of doctors, and tens of thousands of pharmacies, for example. Graph analysis techniques are developed to find suspicious individuals, suspicious relationships between individuals, unusual changes over time, unusual geospatial dispersion, and anomalous networks within the overall graph structure. The system has been deployed on multiple sites and data sets, both government and commercial, to facilitate the work of FWA investigation analysts.
A Bayesian Approach for Policy Learning from Trajectory Preference Queries
Wilson, Aaron, Fern, Alan, Tadepalli, Prasad
We consider the problem of learning control policies via trajectory preference queries to an expert. In particular, the learning agent can present an expert with short runs of a pair of policies originating from the same state and the expert then indicates the preferred trajectory. The agent's goal is to elicit a latent target policy from the expert with as few queries as possible. To tackle this problem we propose a novel Bayesian model of the querying process and introduce two methods that exploit this model to actively select expert queries. Experimental results on four benchmark problems indicate that our model can effectively learn policies from trajectory preference queries and that active query selection can be substantially more efficient than random selection.
Bayesian Policy Search for Multi-Agent Role Discovery
Wilson, Aaron (Oregon State University) | Fern, Alan (Oregon State University) | Tadepalli, Prasad (Oregon State University)
Bayesian inference is an appealing approach for leveraging prior knowledge in reinforcement learning (RL). In this paper we describe an algorithm for discovering different classes of roles for agents via Bayesian inference. In particular, we develop a Bayesian policy search approach for Multi-Agent RL (MARL), which is model-free and allows for priors on policy parameters. We present a novel optimization algorithm based on hybrid MCMC, which leverages both the prior and gradient information estimated from trajectories. Our experiments in a complex real-time strategy game demonstrate the effective discovery of roles from supervised trajectories, the use of discovered roles for successful transfer to similar tasks, and the discovery of roles through reinforcement learning.