Monte Carlo Tree Search (MCTS) is particularly adapted to domains where the potential actions can be represented as a tree of sequential decisions. For an effective action selection, MCTS performs many simulations to build a reliable tree representation of the decision space. As such, a bottleneck to MCTS appears when enough simulations cannot be performed between action selections. This is particularly highlighted in continuously running tasks, for which the time available to perform simulations between actions tends to be limited due to the environment's state constantly changing. In this paper, we present an approach that takes advantage of the anytime characteristic of MCTS to increase the simulation time when allowed. Our approach is to effectively balance the prospect of selecting an action with the time that can be spared to perform MCTS simulations before the next action selection. For that, we considered the simulation time as a decision variable to be selected alongside an action. We extended the Hierarchical Optimistic Optimization applied to Tree (HOOT) method to adapt our approach to environments with a continuous decision space. We evaluated our approach for environments with a continuous decision space through OpenAI gym's Pendulum and Continuous Mountain Car environments and for environments with discrete action space through the arcade learning environment (ALE) platform. The evaluation results show that, with variable simulation times, the proposed approach outperforms the conventional MCTS in the evaluated continuous decision space tasks and improves the performance of MCTS in most of the ALE tasks.

With online calendar services gaining popularity worldwide, calendar data has become one of the richest context sources for understanding human behavior. However, event scheduling is still time-consuming even with the development of online calendars. Although machine learning based event scheduling models have automated scheduling processes to some extent, they often fail to understand subtle user preferences and complex calendar contexts with event titles written in natural language. In this paper, we propose Neural Event Scheduling Assistant (NESA) which learns user preferences and understands calendar contexts, directly from raw online calendars for fully automated and highly effective event scheduling. We leverage over 593K calendar events for NESA to learn scheduling personal events, and we further utilize NESA for multi-attendee event scheduling. NESA successfully incorporates deep neural networks such as Bidirectional Long Short-Term Memory, Convolutional Neural Network, and Highway Network for learning the preferences of each user and understanding calendar context based on natural languages. The experimental results show that NESA significantly outperforms previous baseline models in terms of various evaluation metrics on both personal and multi-attendee event scheduling tasks. Our qualitative analysis demonstrates the effectiveness of each layer in NESA and learned user preferences.

Chance Constrained Markov Decision Processes maximize reward subject to a bounded probability of failure, and have been frequently applied for planning with potentially dangerous outcomes or unknown environments. Solution algorithms have required strong heuristics or have been limited to relatively small problems with up to millions of states, because the optimal action to take from a given state depends on the probability of failure in the rest of the policy, leading to a coupled problem that is difficult to solve. In this paper we examine a generalization of a CCMDP that trades off probability of failure against reward through a functional relationship. We derive a constraint that can be applied to each state history in a policy individually, and which guarantees that the chance constraint will be satisfied. The approach decouples states in the CCMDP, so that large problems can be solved efficiently. We then introduce Vulcan, which uses our constraint in order to apply Monte Carlo Tree Search to CCMDPs. Vulcan can be applied to problems where it is unfeasible to generate the entire state space, and policies must be returned in an anytime manner. We show that Vulcan and its variants run tens to hundreds of times faster than linear programming methods, and over ten times faster than heuristic based methods, all without the need for a heuristic, and returning solutions with a mean suboptimality on the order of a few percent. Finally, we use Vulcan to solve for a chance constrained policy in a CCMDP with over $10^{13}$ states in 3 minutes.

In this paper, the multiple operating room (OR) surgical case sequencing problem (SCSP) is addressed. The objective is to maximise total OR utilisation during standard opening hours. The work here is based on a case study of a large Australian public hospital with long surgical waiting lists and high levels of non-elective demand. Due to the complexity of the SCSP and the size of the instances considered herein, heuristic techniques are required to solve the problem. Constructive heuristics are presented based on both a modified block scheduling policy and an open scheduling policy. A number of real-time reactive strategies are presented that can be used to maintain schedule feasibility in the case of disruptions. Results of computational experiments show that the approach presented in this paper can be used to maintain schedule feasibility in real-time, whilst increasing OT utilisation and throughput, and reducing the waiting time of non-elective patients. The framework presented here is applicable to the real-life scheduling of OT departments, and recommendations have been provided regarding implementation of the approach.

In this paper we consider the surgical case sequencing problem (SCSP) under stochastic conditions. In addition to implementing a robust surgical schedule, we investigate the use of a number of reactive strategies that can be used to maintain schedule feasibility. We present a mixed integer nonlinear programming (MINLP) model for the reactive multiple operating room (OR) SCSP that may be suitable for direct implementation on small problem instances. A machine scheduling perspective is considered and the model is equivalent to a resource-constrained parallel-machine scheduling problem with identical machines, machine eligibility restrictions, and machine and job release dates. The explicit objective of the model is to reduce OR idle time, although other common objectives (including time to surgery and overtime) are discussed. The work here is based on a case study of a large Australian public hospital with long surgical waiting lists and high non-elective demand. Results of computational experiments show that the reactive strategies presented in this paper can be used to reduce idle time without putting excessive pressure on surgeons.

Finite State Controllers (FSCs) are an effective way to compactly represent sequential plans. By imposing appropriate conditions on transitions, FSCs can also represent generalized plans (plans that solve a range of planning problems from a given domain). In this paper we introduce the concept of hierarchical FSCs for planning by allowing controllers to call other controllers. This call mechanism allows hierarchical FSCs to represent generalized plans more compactly than individual FSCs, to compute controllers in a modular fashion or even more, to compute recursive controllers. The paper introduces a classical planning compilation for computing hierarchical FSCs that solve challenging generalized planning tasks. The compilation takes as input a finite set of classical planning problems from a given domain. The output of the compilation is a single classical planning problem whose solution induces: (1) a hierarchical FSC and (2), the corresponding validation of that controller on the input classical planning problems.

Abstract--This work proposes an engine for the Creation Of Novel Adventure Narrative (CONAN), which is a procedural quest generator. It uses a planning approach to story generation. The engine is tested on its ability to create quests, which are sets of actions that must be performed in order to achieve a certain goal, usually for a reward. The engine takes in a world description represented as a set of facts, including characters, locations, and items, and generates quests according to the state of the world and the preferences of the characters. We evaluate quests through the classification of the motivations behind the quests, based on the sequences of actions required to complete the quests. We also compare different world descriptions and analyze the difference in motivations for the quests produced by the engine. Compared against human structural quest analysis, the current engine was found to be able to replicate the quest structures found in commercial video game quests. The creation of media content has always been the domain of humans, be it for movies, music or video games. With advancement in computer technology and research, the creation of such content has seen a slight shift from the human authored to automatic computer generation. Using algorithms to procedurally create media can effectively alleviate some of the burden from artists when creating a new piece. A. Procedural Generation in Games Procedural Content Generation for Games (PCG-G) is the use of computers algorithms to generate game content, determine if it is interesting, and select the best ones on behalf of the players.[1] This type of generation becomes quite useful when trying to produce content for an industry that is more and more demanding in terms of content [1]. For instance, in the current market, game development costs are extremely high as the demand for highly complex games requires the work of many artists and many hours to be met. For instance, the Massively Multiplayer Online Role Playing Game (MMORPG) World of Warcraft has a total of 30,000 items, 5,300 creatures with which to interact and 7600 quests and has an estimated budget of twenty to one hundred and fifty million dollars for a single game [1].

Researchers often try to capture as much information as they can, either by using existing architectures, creating new ones, going deeper, or employing different training methods. This paper compares different ideas and methods that are used heavily in Machine Learning to determine what works best. These methods are prevalent in various domains of Machine Learning, such as Computer Vision and Natural Language Processing (NLP). Throughout our work, we have tried to bring generalization into context, because that's what matters in the end. Any model should be robust and able to work outside your research environment. When a model lacks generalization, very often we try to train the model on datasets it has never encountered … and that's when things start to get much more complex.

The basic argument is increasingly deployed by frustrated executives and self-promoting private-equity groups: Companies are doing dumb things to meet the market's quarterly expectations, and hurting their long-term prospects as a result. Take the company private and executives no longer have to care about the short term, allowing them to invest for the long run and help the company, their loyal shareholders and wider society. The trouble is that none of this applies to Tesla. It is hard to think of a company that cares less about sucking up to Wall Street than Tesla. Mr. Musk earlier this year rejected "boring bonehead questions" from analysts on his quarterly earnings call; the company offers no guidance on quarterly earnings; and it has frequently and unapologetically reported losses far worse than expected (only twice has it made a quarterly profit, both times a surprise).

Abstract-- Driving is a social activity: drivers often indicate their intent to change lanes via motion cues. We consider mixed-autonomy traffic where a Human-driven V ehicle (HV) and an Autonomous V ehicle (A V) drive together . We propose a planning framework where the degree to which the A V considers the other agent's reward is controlled by a selfishness factor . We test our approach on a simulated two-lane highway where the A V and HV merge into each other's lanes. In a user study with 21 subjects and 6 different selfishness factors, we found that our planning approach was sound and that both agents had less merging times when a factor that balances the rewards for the two agents was chosen. Our results on double lane merging suggest it to be a nonzero-sum game and encourage further investigation on collaborative decision making algorithms for mixed-autonomy traffic. Driving is a social activity: drivers indicate their willingness to change lanes by subtle cues such as eye contact, or by not-so-subtle cues such as adjusting their speed and position [1]. There has been impressive demonstrations of Autonomous V ehicle (A V) technology [2]-[4], however one of the remaining challenges in this area is reading those cues to estimate the intentions of other agents as well as using cues to communicate the intentions of the A V . As A Vs become commonplace, the situations where A V's and Human-driven V ehicles (HV) interact will increase.