A graphical model is a structured representation of locally dependent random variables. A traditional method to reason over these random variables is to perform inference using belief propagation. When provided with the true data generating process, belief propagation can infer the optimal posterior probability estimates in tree structured factor graphs. However, in many cases we may only have access to a poor approximation of the data generating process, or we may face loops in the factor graph, leading to suboptimal estimates. In this work we first extend graph neural networks to factor graphs (FG-GNN). We then propose a new hybrid model that runs conjointly a FG-GNN with belief propagation. The FG-GNN receives as input messages from belief propagation at every inference iteration and outputs a corrected version of them. As a result, we obtain a more accurate algorithm that combines the benefits of both belief propagation and graph neural networks. We apply our ideas to error correction decoding tasks, and we show that our algorithm can outperform belief propagation for LDPC codes on bursty channels.

In many sequential decision-making problems, the goal is to optimize a utility function while satisfying a set of constraints on different utilities. This learning problem is formalized through Constrained Markov Decision Processes (CMDPs). In this paper, we investigate the exploration-exploitation dilemma in CMDPs. While learning in an unknown CMDP, an agent should trade-off exploration to discover new information about the MDP, and exploitation of the current knowledge to maximize the reward while satisfying the constraints. While the agent will eventually learn a good or optimal policy, we do not want the agent to violate the constraints too often during the learning process. In this work, we analyze two approaches for learning in CMDPs. The first approach leverages the linear formulation of CMDP to perform optimistic planning at each episode. The second approach leverages the dual formulation (or saddle-point formulation) of CMDP to perform incremental, optimistic updates of the primal and dual variables. We show that both achieves sublinear regret w.r.t.\ the main utility while having a sublinear regret on the constraint violations. That being said, we highlight a crucial difference between the two approaches; the linear programming approach results in stronger guarantees than in the dual formulation based approach.

Time series classification using novel techniques has experienced a recent resurgence and growing interest from statisticians, subject-domain scientists, and decision makers in business and industry. This is primarily due to the ever increasing amount of big and complex data produced as a result of technological advances. A motivating example is that of Google trends data, which exhibit highly nonlinear behavior. Although a rich literature exists for addressing this problem, existing approaches mostly rely on first and second order properties of the time series, since they typically assume linearity of the underlying process. Often, these are inadequate for effective classification of nonlinear time series data such as Google Trends data. Given these methodological deficiencies and the abundance of nonlinear time series that persist among real-world phenomena, we introduce an approach that merges higher order spectral analysis (HOSA) with deep convolutional neural networks (CNNs) for classifying time series. The effectiveness of our approach is illustrated using simulated data and two motivating industry examples that involve Google trends data and electronic device energy consumption data.

In cooperative multi-agent reinforcement learning (MARL), how to design a suitable reward signal to accelerate learning and stabilize convergence is a critical problem. The global reward signal assigns the same global reward to all agents without distinguishing their contributions, while the local reward signal provides different local rewards to each agent based solely on individual behavior. Both of the two reward assignment approaches have some shortcomings: the former might encourage lazy agents, while the latter might produce selfish agents. In this paper, we study reward design problem in cooperative MARL based on packet routing environments. Firstly, we show that the above two reward signals are prone to produce suboptimal policies. Then, inspired by some observations and considerations, we design some mixed reward signals, which are off-the-shelf to learn better policies. Finally, we turn the mixed reward signals into the adaptive counterparts, which achieve best results in our experiments. Other reward signals are also discussed in this paper. As reward design is a very fundamental problem in RL and especially in MARL, we hope that MARL researchers can rethink the rewards used in their systems.

Internet of things (IoT) devices are becoming increasingly popular thanks to many new services and applications they offer. However, in addition to their many benefits, they raise privacy concerns since they share fine-grained time-series user data with untrusted third parties. In this work, we study the privacy-utility trade-off (PUT) in time-series data sharing. Existing approaches to PUT mainly focus on a single data point; however, temporal correlations in time-series data introduce new challenges. Methods that preserve the privacy for the current time may leak significant amount of information at the trace level as the adversary can exploit temporal correlations in a trace. We consider sharing the distorted version of a user's true data sequence with an untrusted third party. We measure the privacy leakage by the mutual information between the user's true data sequence and shared version. We consider both instantaneous and average distortion between the two sequences, under a given distortion measure, as the utility loss metric. To tackle the history-dependent mutual information minimization, we reformulate the problem as a Markov decision process (MDP), and solve it using asynchronous actor-critic deep reinforcement learning (RL). We apply our optimal data release policies to location trace privacy scenario, and evaluate the performance of the proposed policy numerically.

Someday soon, oil refineries may trade in crude oil for agricultural waste like corn stalks or renewable plants like switchgrass in order to produce sustainable biofuels. But we're not there quite yet; converting those products into usable chemicals on a large scale requires efficient catalytic reactions, which researchers are still hunting for. Recently, Conway Assistant Professor Reid Van Lehn and his colleagues in the Department of Chemical and Biological Engineering have found a way to speed up the process of finding suitable reaction conditions using machine learning, which may help the era of biofuels come a little bit sooner. One of the ways to convert lignocellulosic biomass into usable fuels is via acid-catalyzed reactions, which usually take place in water. It's often a slow process, but research has shown that the addition of certain organic cosolvents can increase reaction rates 100-fold or more.

Artificial Intelligence (AI) can be applied to a lot of industrial environments to save costs and to improve processes. This industrial Artificial Intelligence does not only include the smart algorithms and Big Data concepts that reside in the virtual space inside the computer systems, but it consists of the physical devices themselves too. Data has to be captured with sensors. Commands have to be sent to actuators and control systems. This whole chain and flow of information, wireless or via cables, goes through places with extreme conditions.

Reduced-order models that accurately abstract high fidelity models and enable faster simulation is vital for real-time, model-based diagnosis applications. In this paper, we outline a novel hybrid modeling approach that combines machine learning inspired models and physics-based models to generate reduced-order models from high fidelity models. We are using such models for real-time diagnosis applications. Specifically, we have developed machine learning inspired representations to generate reduced order component models that preserve, in part, the physical interpretation of the original high fidelity component models. To ensure the accuracy, scalability and numerical stability of the learning algorithms when training the reduced-order models we use optimization platforms featuring automatic differentiation. Training data is generated by simulating the high-fidelity model. We showcase our approach in the context of fault diagnosis of a rail switch system. Three new model abstractions whose complexities are two orders of magnitude smaller than the complexity of the high fidelity model, both in the number of equations and simulation time are shown. The numerical experiments and results demonstrate the efficacy of the proposed hybrid modeling approach.

Pattern similarity-based methods are widely used in classification and regression problems. Repeated, similar-shaped cycles observed in seasonal time series encourage to apply these methods for forecasting. In this paper we use the pattern similarity-based methods for forecasting monthly electricity demand expressing annual seasonality. An integral part of the models is the time series representation using patterns of time series sequences. Pattern representation ensures the input and output data unification through trend filtering and variance equalization. Consequently, pattern representation simplifies the forecasting problem and allows us to use models based on pattern similarity. We consider four such models: nearest neighbor model, fuzzy neighborhood model, kernel regression model and general regression neural network. A regression function is constructed by aggregation output patterns with weights dependent on the similarity between input patterns. The advantages of the proposed models are: clear principle of operation, small number of parameters to adjust, fast optimization procedure, good generalization ability, working on the newest data without retraining, robustness to missing input variables, and generating a vector as an output. In the experimental part of the work the proposed models were used to forecasting the monthly demand for 35 European countries. The model performances were compared with the performances of the classical models such as ARIMA and exponential smoothing as well as state-of-the-art models such as multilayer perceptron, neuro-fuzzy system and long short-term memory model. The results show high performance of the proposed models which outperform the comparative models in accuracy, simplicity and ease of optimization.

Alphabet's R&D subsidiary X has unveiled its next "moonshot project," as the company strives to improve ocean health through automatically tracking and logging the behavior of fish over time. The oceans constitute more than 70% of Earth's surface, and play a pivotal part in regulating the global climate. They also provides billions of people with food, with fish generally having a lower carbon footprint relative to land-based animals. But understanding what's going on underneath the oceans is key to protecting it, which is what Google's sister company is now setting out to achieve. Tidal, as the new project is called, is setting out to help preserve the ocean's ability to "support life and help feed humanity, sustainably," noted Tidal project lead Neil Davé, in a blog post.