Finite State Controllers (FSCs) are an effective way to represent sequential plans compactly. By imposing appropriate conditions on transitions, FSCs can also represent generalized plans that solve a range of planning problems from a given domain. In this paper we introduce the concept of hierarchical FSCs for planning by allowing controllers to call other controllers. We show that hierarchical FSCs can represent generalized plans more compactly than individual FSCs. Moreover, our call mechanism makes it possible to generate hierarchical FSCs in a modular fashion, or even to apply recursion. We also introduce a compilation that enables a classical planner to generate hierarchical FSCs that solve challenging generalized planning problems. The compilation takes as input a set of planning problems from a given domain and outputs a single classical planning problem, whose solution corresponds to a hierarchical FSC. 1 Introduction Finite state controllers (FSCs) are a compact and effective representation commonly used in AI; prominent examples include robotics [ Brooks, 1989 ] and video-games [ Buckland, 2004] . In planning, FSCs offer two main benefits: 1) solution compactness [ B ackstr om et al., 2014 ]; and 2) the ability to represent generalized plans that solve a range of similar planning problems. This generalization capacity allows FSCs to represent solutions to arbitrarily large problems, as well as problems with partial observability and non-deterministic actions [ Bonet et al., 2010; Hu and Levesque, 2011; Srivastava et al., 2011; Hu and De Giacomo, 2013 ] .

Path Planning and target searching in a three-dimensional environment is a challenging task in the field of robotics. It is an optimization problem as the path from source to destination has to be optimal. This paper aims to generate a collision-free trajectory in a dynamic environment. The path planning problem has sought to be of extreme importance in the military, search and rescue missions and in life-saving tasks. During its operation, the unmanned air vehicle operates in a hostile environment, and faster replanning is needed to reach the target as optimally as possible. This paper presents a novel approach of hierarchical planning using multiresolution abstract levels for faster replanning. Economic constraints like path length, total path planning time and the number of turns are taken into consideration that mandate the use of cost functions. Experimental results show that the hierarchical version of GSO gives better performance compared to the BBO, IWO and their hierarchical versions.

Simulations of infectious disease spread have long been used to understand how epidemics evolve and how to effectively treat them. However, comparatively little attention has been paid to understanding the fairness implications of different treatment strategies -- that is, how might such strategies distribute the expected disease burden differentially across various subgroups or communities in the population? In this work, we define the precision disease control problem -- the problem of optimally allocating vaccines in a social network in a step-by-step fashion -- and we use the ML Fairness Gym to simulate epidemic control and study it from both an efficiency and fairness perspective. We then present an exploratory analysis of several different environments and discuss the fairness implications of different treatment strategies.

We consider Monte-Carlo Tree Search (MCTS) applied to Markov Decision Processes (MDPs) and Partially Observable MDPs (POMDPs), and the well-known Upper Confidence bound for Trees (UCT) algorithm. In UCT, a tree with nodes (states) and edges (actions) is incrementally built by the expansion of nodes, and the values of nodes are updated through a backup strategy based on the average value of child nodes. However, it has been shown that with enough samples the maximum operator yields more accurate node value estimates than averaging. Instead of settling for one of these value estimates, we go a step further proposing a novel backup strategy which uses the power mean operator, which computes a value between the average and maximum value. We call our new approach Power-UCT and argue how the use of the power mean operator helps to speed up the learning in MCTS. We theoretically analyze our method providing guarantees of convergence to the optimum. Moreover, we discuss a heuristic approach to balance the greediness of backups by tuning the power mean operator according to the number of visits to each node. Finally, we empirically demonstrate the effectiveness of our method in well-known MDP and POMDP benchmarks, showing significant improvement in performance and convergence speed w.r.t. UCT.

Path planning is a fundamental and extensively explored problem in robotic control. We present a novel economic perspective on path planning. Specifically, we investigate strategic interactions among path planning agents using a game theoretic path planning framework. Our focus is on economic tension between two important objectives: efficiency in the agents' achieving their goals, and safety in navigating towards these. We begin by developing a novel mathematical formulation for path planning that trades off these objectives, when behavior of other agents is fixed. We then use this formulation for approximating Nash equilibria in path planning games, as well as to develop a multi-agent cooperative path planning formulation. Through several case studies, we show that in a path planning game, safety is often significantly compromised compared to a cooperative solution.

Connected devices and the subsequent birth of the Internet of Things (IoT) have provided businesses a veritable treasure trove of data to mine, understand, and reapply to their own processes. And thus, the Industrial Internet of Things (IIoT) was born. Most enterprises today have only scratched the surface of what their connected device data can do for them. This is especially true for manufacturers, who have endless pieces of equipment, product and supply chains to manage on a day-to-day basis. In today's competitive landscape, there is a great opportunity for manufacturing companies to use the data from their connected devices and adopt and apply machine learning solutions in order to help solve their longstanding pain points of resource planning, machine maintenance, and supply chain management.

Q: Peter, for those who haven't heard of it, what is the best way to describe BoldIQ? A: At BoldIQ, we work towards an entirely demand-driven world, uninhibited by the constraints of supply, timing and disruption. This is a world where everything is available when and where customers want, and our role is to provide organizations with state-of-the-art technology to successfully deliver on that demand. BoldIQ's Solver is planning and scheduling optimization software that was initially developed by a private air charter company, and BoldIQ subsequently acquired that expertise. With Solver's success over the last ten years in the private air charter industry, this next-generation solution now also delivers advanced planning and scheduling for other mission-critical, cost-sensitive industries such as logistics, healthcare, modern transportation, construction and others.

Finding the optimum path for a robot for moving from start to the goal position through obstacles is still a challenging issue. Thi s paper presents a novel path planning method, named D - point trigonometric, based on Q - learning algorithm for dynamic and uncertain environments, in which all the obstacles and the target are moving. We define a new state, action and reward functions for t he Q - learning by which the agent can find the best action in every state to reach the goal in the most appropriate path. Moreover, the experiment s in Unity3D confirmed the high convergence speed, the high hit rate, as well as the low dependency on environmental parameters of the proposed method compared with an opponent approach. The planning has been considered as a challenging concern in video games [1], transportation systems [2], and mobile robots [3] [4] . A s the most important path planning issues, w e can refer to the dynamics and the uncertainty of the environment, the smoothness and the length of the path, obstacle avoidance, and the computation al cost . In the last few decades, researchers have done numerous research efforts to present new approaches to solve them [5] [6] [7] [8] . Generally, most of the path planning approaches are categorized to one of the following methods [9] [10] [11]: ( 1) Classical methods (a) Computational geometry (CG) (b) Probabilistic r oadmap (PRM) (c) Potential fields method (PFM) ( 2) Heuristic and meta heuristic methods (a) Soft computing (b) Hybrid algorithms Since the complexity and the execution time of CG methods were high [11], PRMs were proposed to red uce the search space using techniques like milestones [12] .

Artificial intelligence (AI) continues to draw a lot of attention as companies and technology vendors look at how machine learning could improve supply chain operations. In particular demand planning, understood here as the process of developing forecasts that will drive operational supply chain decisions, is being touted as the next potential field for innovation. Technology giants like Amazon and Microsoft have announced AI tools for improving demand planning, and several consulting companies are promoting their skills to bring AI to companies' demand planning processes. In fact, a recent survey by the Institute of Business Forecasting and Planning (IBF) identified AI as the technology that will have the largest impact on demand planning in the next seven years.1 It's not hard to see the fit between AI and demand planning. Demand planning involves lots of number crunching and data analytics, and it is repeated cycle after cycle.

Most common navigation tasks in human environments require auxiliary arm interactions, e.g. opening doors, pressing buttons and pushing obstacles away. This type of navigation tasks, which we call Interactive Navigation, requires the use of mobile manipulators: mobile bases with manipulation capabilities. Interactive Navigation tasks are usually long-horizon and composed of heterogeneous phases of pure navigation, pure manipulation, and their combination. Using the wrong part of the embodiment is inefficient and hinders progress. We propose HRL4IN, a novel Hierarchical RL architecture for Interactive Navigation tasks. HRL4IN exploits the exploration benefits of HRL over flat RL for long-horizon tasks thanks to temporally extended commitments towards subgoals. Different from other HRL solutions, HRL4IN handles the heterogeneous nature of the Interactive Navigation task by creating subgoals in different spaces in different phases of the task. Moreover, HRL4IN selects different parts of the embodiment to use for each phase, improving energy efficiency. We evaluate HRL4IN against flat PPO and HAC, a state-of-the-art HRL algorithm, on Interactive Navigation in two environments - a 2D grid-world environment and a 3D environment with physics simulation. We show that HRL4IN significantly outperforms its baselines in terms of task performance and energy efficiency. More information is available at https://sites.google.com/view/hrl4in.