We study the safe reinforcement learning problem with nonlinear function approximation, where policy optimization is formulated as a constrained optimization problem with both the objective and the constraint being nonconvex functions. For such a problem, we construct a sequence of surrogate convex constrained optimization problems by replacing the nonconvex functions locally with convex quadratic functions obtained from policy gradient estimators. We prove that the solutions to these surrogate problems converge to a stationary point of the original nonconvex problem. Furthermore, to extend our theoretical results, we apply our algorithm to examples of optimal control and multi-agent reinforcement learning with safety constraints.

We have nearly 30 years' experience in research and teaching. Our efforts focus on four complimentary fields: Artificial Intelligence, Data Science, Computer Science and Applied Mathematics. The department's contributions to areas such as multi-agent systems, (medical) signal and image processing, game theory and AI search techniques are internationally recognized. DKE maintains a large and worldwide network of public and private collaborators, and our staff is firmly rooted in multiple national and international research networks. Next to our research, we take pride in our education.

A bank's AI goals should look beyond cost reduction, welcome though that is. This technology has the capacity to do much more. Indeed, it can completely transform on institution from the core." In our recent Banking on AI article, we gave an overview of the rapid pace of AI evolution in the banking sector and looked at the limitless opportunity to automate functions and augment the banking workplace. Microsft's Dynamics 365 AI solutions feature the most impactful and proprietary AI capabilities in global technology, and support banking organisations to deliver meaningful customer experience.

The research on coalitional games has focused on how to share the reward among a coalition such that players are in-centivised to collaborate together. It assumes that the (deterministic or stochastic) characteristic function is known in advance. This paper studies a new setting (a task allocation problem) where the characteristic function is not known and it is controlled by some private information from the players. Hence, the challenge here is twofold: (i) incentivize players to reveal their private information truthfully, (ii) incentivize them to collaborate together. We show that existing reward distribution mechanisms or auctions cannot solve the challenge. Hence, we propose the very first mechanism for the problem from the perspective of both mechanism design and coalitional games.

Our research aims to develop intelligent collaborative agents that are human-aware - they can model, learn, and reason about their human partner's physiological, cognitive, and affective states. In this paper, we study how adaptive coaching interactions can be designed to help people develop sustainable healthy behaviors. We leverage the common model of cognition - CMC [26] - as a framework for unifying several behavior change theories that are known to be useful in human-human coaching. We motivate a set of interactive system desiderata based on the CMC-based view of behavior change. Then, we propose PARCoach - an interactive system that addresses the desiderata. PARCoach helps a trainee pick a relevant health goal, set an implementation intention, and track their behavior. During this process, the trainee identifies a specific goal-directed behavior as well as the situational context in which they will perform it. PARCcoach uses this information to send notifications to the trainee, reminding them of their chosen behavior and the context. We report the results from a 4-week deployment with 60 participants. Our results support the CMC-based view of behavior change and demonstrate that the desiderata for proposed interactive system design is useful in producing behavior change.

In Towards a theory of superminds, I describe a theory of collective intelligence based on the active inference framework pioneered by Karl Friston. As required by active inference, that theory implies that all collectives (such as teams and organizations) operate on the basis of an implicit collective model of the world -- which provides them with the ability to make sense of observations and to predict outcomes of alternative courses of action ("policies" in Friston-speak). If we buy into the theorem that any Markov blanket (causally atomic subsystem) can always be described as performing some form of approximate active inference, this isn't polemical at all. Yet, as I've come to understand, there is something deeply counterintuitive in the conclusion. The fact that needs to be explained is that, unlike people, animals, unicellular beings, and even artificial agents, collectives are composed of clearly distinct and autonomous constituents -- and yet are capable of acting as collectives. This collective action can be as simple as the coherent motion of a flock of birds, or as complex as the financial markets incorporating sophisticated information about expectations of the future into asset prices (or for that matter, a corporation acting out a complex business strategy and production structure); the salient point is that the constituents are autonomous (and, at least for the examples involving humans, will tell you they follow their own free will), and yet the collective can be very accurately described as an agent in its own right. It turns out that fully resolving this apparent contradiction requires gaining a thorough understanding of what these models are, which I will attempt to do in this post.

In Towards a theory of superminds, I describe a theory of collective intelligence based on the active inference framework pioneered by Karl Friston. As required by active inference, that theory implies that all collectives (such as teams and organizations) operate on the basis of an implicit collective model of the world -- which provides them with the ability to make sense of observations and to predict outcomes of alternative courses of action ("policies" in Friston-speak). If we buy into the theorem that any Markov blanket (causally atomic subsystem) can always be described as performing some form of approximate active inference, this isn't polemical at all. Yet, as I've come to understand, there is something deeply counterintuitive in the conclusion. The fact that needs to be explained is that, unlike people, animals, unicellular beings, and even artificial agents, collectives are composed of clearly distinct and autonomous constituents -- and yet are capable of acting as collectives. This collective action can be as simple as the coherent motion of a flock of birds, or as complex as the financial markets incorporating sophisticated information about expectations of the future into asset prices (or for that matter, a corporation acting out a complex business strategy and production structure); the salient point is that the constituents are autonomous (and, at least for the examples involving humans, will tell you they follow their own free will), and yet the collective can be very accurately described as an agent in its own right. It turns out that fully resolving this apparent contradiction requires gaining a thorough understanding of what these models are, which I will attempt to do in this post.

Reinforcement learning is a promising approach to learning control policies for performing complex multi-agent robotics tasks. However, a policy learned in simulation often fails to guarantee even simple safety properties such as obstacle avoidance. To ensure safety, we propose multi-agent model predictive shielding (MAMPS), an algorithm that provably guarantees safety for an arbitrary learned policy. In particular, it operates by using the learned policy as often as possible, but instead uses a backup policy in cases where it cannot guarantee the safety of the learned policy. Using a multi-agent simulation environment, we show how MAMPS can achieve good performance while ensuring safety.

Nature provides us with abundant examples of how large numbers of individuals can make decisions without the coordination of a central authority. Social insects, birds, fishes, and many other living collectives, rely on simple interaction mechanisms to do so. They individually gather information from the environment; small bits of a much larger picture that are then shared locally among the members of the collective and processed together to output a commonly agreed choice. Throughout evolution, Nature found solutions to collective decision-making problems that are intriguing to engineers for their robustness to malfunctioning or lost individuals, their flexibility in face of dynamic environments, and their ability to scale with large numbers of members. In the last decades, whereas biologists amassed large amounts of experimental evidence, engineers took inspiration from these and other examples to design distributed algorithms that, while maintaining the same properties of their natural counterparts, come with guarantees on their performance in the form of predictive mathematical models. In this paper, we review the fundamental processes that lead to a collective decision. We discuss examples of collective decisions in biological systems and show how similar processes can be engineered to design artificial ones. During this journey, we review a framework to design distributed decision-making algorithms that are modular, can be instantiated and extended in different ways, and are supported by a suit of predictive mathematical models.

Machine learning algorithms--also known as artificial intelligence (AI)--can better detect sleep apnea compared with traditional linear approaches, according to a study being presented at the CHEST Annual Meeting 2019. The researchers included 620 patients who were referred to a sleep lab in a suburban community sleep center. Researchers collected information on 12 select parameters: height, weight, waist, hip, body mass index, age, neck side, Modified Friedman stage, snoring, Epworth sleepiness scale, sex, and daytime sleepiness. During phase I, researchers used a binary particle swarm optimization technique to select the best sub-features that characterize sleep apnea. In phase II, they built an artificial neural network model based on a feedforward algorithm to detect sleep apnea.