This paper presents a new multi-layered algorithm for motion planning under motion and sensing uncertainties for Linear Temporal Logic specifications. We propose a technique to guide a sampling-based search tree in the combined task and belief space using trajectories from a simplified model of the system, to make the problem computationally tractable. Our method eliminates the need to construct fine and accurate finite abstractions. We prove correctness and probabilistic completeness of our algorithm, and illustrate the benefits of our approach on several case studies. Our results show that guidance with a simplified belief space model allows for significant speed-up in planning for complex specifications.

The Common Information (CI) approach provides a systematic way to transform a multi-agent stochastic control problem to a single-agent partially observed Markov decision problem (POMDP) called the coordinator's POMDP. However, such a POMDP can be hard to solve due to its extraordinarily large action space. We propose a new algorithm for multi-agent stochastic control problems, called coordinator's heuristic search value iteration (CHSVI), that combines the CI approach and point-based POMDP algorithms for large action spaces. We demonstrate the algorithm through optimally solving several benchmark problems.

MLconf gathers machine learning & AI enthusiasts from a broad range of industries and academic backgrounds to share new tools, tricks, platforms, algorithms and methods with a broad audience of practitioners. Each presentation offers an educational component to be shared with the community, in which specific algorithms and techniques can be shared and new applications of such are inspired. Today, we are making a call for presentations for our MLconf San Francisco conference to be held on October 19, 2023 at the Hotel Nikko in SF. The conference will feature presentations from across the machine learning landscape. If you, your team, organization, or colleague has done something innovative related to ML algorithms, Tools and Platforms, or Building and Managing Teams to solve hard problems, let us help you share your story. In your abstract, we encourage you to mention where you feel your techniques will transfer over into other Machine Learning applications, showing where it's relevant to the MLconf audience. Prior submissions have included presentations related to: Algorithms that have graduated from an academic/theory state and have proven to be effective, robust and scalable in production within industry application; Machine Learning/AI examples of specific challenges faced within current industry and how teams have found success by applying new algorithms and techniques or by applying modifications to existing practices for optimal outcomes; New platforms, tools for machine learning; New business practices for managing and growing data science teams; and Expanding machine learning to new domains. Abstracts should be 150-500 words in length and should illustrate the level of technicality in the proposed presentation. At the time of the event, presentations will be generally limited to 25-30 minutes in length in order to allow you to provide depth while also allowing for presentations from colleagues and Q&A. Emphasis should be given to the technical challenges, benchmarks, innovations and motivation for the development of models, algorithms and statistical models to analyze and draw inferences from patterns in data. Your presentation should definitely not be a product or sales pitch. ABSTRACT DEADLINE: June 30, 2023 Topics we are looking for include but are not limited to: AI/ML Ops Natural Language Processing Deep Learning Reinforcement Learning Data Science for Social Good Kernel Methods Causality Embeddings Recommendation Systems Quantum Computing and AI/ML Chemistry & AI/ML Pandemic Data & ML Model Interpretability Fraud Detection DeepFake Detection Generative Teaching Networks Facial Recognition/Biometric Identification Genetics & ML Experimental Reproducibility Best Practices Model Uncertainty and Data Drift Generative Adversarial Networks Transfer Learning Adversarial Machine Learning IoT and edge computing applications Genetic Algorithms Tensor Algebra Probabilistic Programming and Logic Machine Learning for Music and Art Bayesian Methods Markov Logic Networks Synthetic Art, Biology Ethics in Machine Learning Data / Algorithm Ethics Sketching Randomized Algorithms AI Education Game Theory Diversity in AI Community Detection Time Series Image Analysis Structured Learning using Neural Networks Healthcare & ML (Clinical Decision Support Systems, Record Keeping, Medical Imaging, etc.) FinTech & ML (Algorithmic Trading, Predictive Analytics, Fraud Detection & Prevention, Payments, etc.) In the spirit of sharing knowledge, presentation slides are shared with attendees and photographs and/or video footage of presentations are shared as well.

We study entropy-regularized constrained Markov decision processes (CMDPs) under the soft-max parameterization, in which an agent aims to maximize the entropy-regularized value function while satisfying constraints on the expected total utility. By leveraging the entropy regularization, our theoretical analysis shows that its Lagrangian dual function is smooth and the Lagrangian duality gap can be decomposed into the primal optimality gap and the constraint violation. Furthermore, we propose an accelerated dual-descent method for entropy-regularized CMDPs. We prove that our method achieves the global convergence rate $\widetilde{\mathcal{O}}(1/T)$ for both the optimality gap and the constraint violation for entropy-regularized CMDPs. A discussion about a linear convergence rate for CMDPs with a single constraint is also provided.

Interval Markov Decision Processes (IMDPs) are finite-state uncertain Markov models, where the transition probabilities belong to intervals. Recently, there has been a surge of research on employing IMDPs as abstractions of stochastic systems for control synthesis. However, due to the absence of algorithms for synthesis over IMDPs with continuous action-spaces, the action-space is assumed discrete a-priori, which is a restrictive assumption for many applications. Motivated by this, we introduce continuous-action IMDPs (caIMDPs), where the bounds on transition probabilities are functions of the action variables, and study value iteration for maximizing expected cumulative rewards. Specifically, we decompose the max-min problem associated to value iteration to $|\mathcal{Q}|$ max problems, where $|\mathcal{Q}|$ is the number of states of the caIMDP. Then, exploiting the simple form of these max problems, we identify cases where value iteration over caIMDPs can be solved efficiently (e.g., with linear or convex programming). We also gain other interesting insights: e.g., in certain cases where the action set $\mathcal{A}$ is a polytope, synthesis over a discrete-action IMDP, where the actions are the vertices of $\mathcal{A}$, is sufficient for optimality. We demonstrate our results on a numerical example. Finally, we include a short discussion on employing caIMDPs as abstractions for control synthesis.

We extend the provably convergent Full Gradient DQN algorithm for discounted reward Markov decision processes from Avrachenkov et al. (2021) to average reward problems. We experimentally compare widely used RVI Q-Learning with recently proposed Differential Q-Learning in the neural function approximation setting with Full Gradient DQN and DQN. We also extend this to learn Whittle indices for Markovian restless multi-armed bandits. We observe a better convergence rate of the proposed Full Gradient variant across different tasks.

The Boltzmann Machine has been introduced as a means to perform global optimization for multimodal objective functions using the principles of simulated annealing. In this paper we consider its utility as a spurious-free content-addressable memory, and provide bounds on its performance in this context. We show how to exploit the machine's ability to escape local minima, in order to use it, at a constant temperature, for unambiguous associative pattern-retrieval in noisy environments. An association rule, which creates a sphere of influence around each stored pattern, is used along with the Machine's dynamics to match the machine's noisy input with one of the pre-stored patterns. Spurious fIxed points, whose regions of attraction are not recognized by the rule, are skipped, due to the Machine's fInite probability to escape from any state.

Hidden Markov models are widely used for automatic speech recog(cid:173) nition. They inherently incorporate the sequential character of the speech signal and are statistically trained. However, the a-priori choice of the model topology limits their flexibility. Another draw(cid:173) back of these models is their weak discriminating power. Multilayer perceptrons are now promising tools in the connectionist approach for classification problems and have already been successfully tested on speech recognition problems. However, the sequential nature of the speech signal remains difficult to handle in that kind of ma(cid:173) chine.

The concept of the stochastic Boltzmann machine (BM) is auractive for decision making and pattern classification purposes since the probability of attaining the network states is a function of the network energy. Hence, the probability of attaining particular energy minima may be associated with the probabilities of making certain decisions (or classifications). However, because of its stochastic nature, the complexity of the BM is fairly high and therefore such networks are not very likely to be used in practice. In this paper we suggest a way to alleviate this drawback by converting the sto(cid:173) chastic BM into a deterministic network which we call the Boltzmann Per(cid:173) ceptron Network (BPN). The BPN is functionally equivalent to the BM but has a feed-forward structure and low complexity. The conditions under which such a convmion is feasible are given.

We are developing a phoneme based. In [Bourlard & Wellekens]. it was shown that MLPs were approximating Maximum a Posteriori (MAP) probabilities and could thus be embedded as an emission probability estimator in HMMs. It is shown here that word recognition performance for a simple discrete density HMM system appears to be somewhat better when MLP methods are used to estimate the emission probabilities.