General conversational intelligence appears to be an important part of artificial general intelligence. Respectively, it requires accessible measures of the intelligence quality and controllable ways of its achievement, ideally - having the linguistic and semantic models represented in a reasonable way. Our work is suggesting to use "Baby Turing Test" approach to extend the classic "Turing Test" for conversational intelligence and controlled language based on semantic graph representation extensible for arbitrary subject domain. We describe how the two can be used together to build a general-purpose conversational system such as intelligent assistant for online media and social network data processing.

Exploration of the high-dimensional state action space is one of the biggest challenges in Reinforcement Learning (RL), especially in multi-agent domain. We present a novel technique called Experience Augmentation, which enables a time-efficient and boosted learning based on a fast, fair and thorough exploration to the environment. It can be combined with arbitrary off-policy MARL algorithms and is applicable to either homogeneous or heterogeneous environments. We demonstrate our approach by combining it with MADDPG and verifing the performance in two homogeneous and one heterogeneous environments. In the best performing scenario, the MADDPG with experience augmentation reaches to the convergence reward of vanilla MADDPG with 1/4 realistic time, and its convergence beats the original model by a significant margin. Our ablation studies show that experience augmentation is a crucial ingredient which accelerates the training process and boosts the convergence.

Intrusion detection is one of the primary mechanisms to provide computer networks with security. With an increase in attacks and growing dependence on various fields such as medicine, commercial, and engineering to give services over a network, securing networks have become a significant issue. The purpose of Intrusion Detection Systems (IDS) is to make models which can recognize regular communications from abnormal ones and take necessary actions. Among different methods in this field, Artificial Neural Networks (ANNs) have been widely used. However, ANN-based IDS, has two main disadvantages: 1- Low detection precision. 2- Weak detection stability. To overcome these issues, this paper proposes a new approach based on Deep Neural Network (DNN. The general mechanism of our model is as follows: first, some of the data in dataset is properly ranked, afterwards, dataset is normalized with Min-Max normalizer to fit in the limited domain. Then dimensionality reduction is applied to decrease the amount of both useless dimensions and computational cost. After the preprocessing part, Mean-Shift clustering algorithm is the used to create different subsets and reduce the complexity of dataset. Based on each subset, two models are trained by Support Vector Machine (SVM) and deep learning method. Between two models for each subset, the model with a higher accuracy is chosen. This idea is inspired from philosophy of divide and conquer. Hence, the DNN can learn each subset quickly and robustly. Finally, to reduce the error from the previous step, an ANN model is trained to gain and use the results in order to be able to predict the attacks. We can reach to 95.4 percent of accuracy. Possessing a simple structure and less number of tunable parameters, the proposed model still has a grand generalization with a high level of accuracy in compared to other methods such as SVM, Bayes network, and STL.

Reinforcement learning (RL) algorithms find applications in inventory control, recommender systems, vehicular traffic management, cloud computing and robotics. The real-world complications of many tasks arising in these domains makes them difficult to solve with the basic assumptions underlying classical RL algorithms. RL agents in these applications often need to react and adapt to changing operating conditions. A significant part of research on single-agent RL techniques focuses on developing algorithms when the underlying assumption of stationary environment model is relaxed. This paper provides a survey of RL methods developed for handling dynamically varying environment models. The goal of methods not limited by the stationarity assumption is to help autonomous agents adapt to varying operating conditions. This is possible either by minimizing the rewards lost during learning by RL agent or by finding a suitable policy for the RL agent which leads to efficient operation of the underlying system. A representative collection of these algorithms is discussed in detail in this work along with their categorization and their relative merits and demerits. Additionally we also review works which are tailored to application domains. Finally, we discuss future enhancements for this field.

Particularly in Natural Language Processing (NLP), the necessity of a good representation of the words that achieves some implicit context understanding is important [Turian et al., 2010]. A typical representation to feed in a machine learning model is the binary one-hot vector, in this case, an array with as many positions as words in the vocabulary is created, and the words are represented by a version of the array containing a one digit in the position corresponding to the word. For example, the sentence "I like eating bread and eating cheese", would have as vocabulary the set "I", "like", "eating", "bread", "and", "cheese", thus the representation of this words would be 6-dimensional binary one-hot vectors like "I" 100000, "like" 010000, cheese 000001. As you can imagine, this representation has no context understanding at all, since all words are completely independent, "bread" and "cheese" are as different between them as "I" and "like", which for a human is not the case.

This paper surveys deep learning algorithms, IoT cyber security and risk models, and established mathematical formulas to identify the best approach for developing a dynamic and self-adapting system for predictive cyber risk analytics supported with Artificial Intelligence and Machine Learning and real-time intelligence in edge computing. The paper presents a new mathematical approach for integrating concepts for cognition engine design, edge computing and Artificial Intelligence and Machine Learning to automate anomaly detection. This engine instigates a step change by applying Artificial Intelligence and Machine Learning embedded at the edge of IoT networks, to deliver safe and functional real-time intelligence for predictive cyber risk analytics. This will enhance capacities for risk analytics and assists in the creation of a comprehensive and systematic understanding of the opportunities and threats that arise when edge computing nodes are deployed, and when Artificial Intelligence and Machine Learning technologies are migrated to the periphery of the internet and into local IoT networks.

This thesis describes work on two applications of probabilistic programming: the learning of probabilistic program code given specifications, in particular program code of one-dimensional samplers; and the facilitation of sequential Monte Carlo inference with help of data-driven proposals. The latter is presented with experimental results on a linear Gaussian model and a non-parametric dependent Dirichlet process mixture of objects model for object recognition and tracking. In Chapter 1 we provide a brief introduction to probabilistic programming. In Chapter 2 we present an approach to automatic discovery of samplers in the form of probabilistic programs. We formulate a Bayesian approach to this problem by specifying a grammar-based prior over probabilistic program code. We use an approximate Bayesian computation method to learn the programs, whose executions generate samples that statistically match observed data or analytical characteristics of distributions of interest. In our experiments we leverage different probabilistic programming systems to perform Markov chain Monte Carlo sampling over the space of programs. Experimental results have demonstrated that, using the proposed methodology, we can learn approximate and even some exact samplers. Finally, we show that our results are competitive with regard to genetic programming methods. In Chapter 3, we describe a way to facilitate sequential Monte Carlo inference in probabilistic programming using data-driven proposals. In particular, we develop a distance-based proposal for the non-parametric dependent Dirichlet process mixture of objects model. We implement this approach in the probabilistic programming system Anglican, and show that for that model data-driven proposals provide significant performance improvements. We also explore the possibility of using neural networks to improve data-driven proposals.

In scenarios that require teamwork, we usually have at hand a variety of specific tasks, for which we need to form a team in order to carry out each one. Here we target the problem of matching teams with tasks within the context of education, and specifically in the context of forming teams of students and allocating them to internship programs. First we provide a formalization of the Team Allocation for Internship Programs Problem, and show the computational hardness of solving it optimally. Thereafter, we propose TAIP, a heuristic algorithm that generates an initial team allocation which later on attempts to improve in an iterative process. Moreover, we conduct a systematic evaluation to show that TAIP reaches optimality, and outperforms CPLEX in terms of time.

The goal of this work is to provide a viable solution based on reinforcement learning for traffic signal control problems. Although the state-of-the-art reinforcement learning approaches have yielded great success in a variety of domains, directly applying it to alleviate traffic congestion can be challenging, considering the requirement of high sample efficiency and how training data is gathered. In this work, we address several challenges that we encountered when we attempted to mitigate serious traffic congestion occurring in a metropolitan area. Specifically, we are required to provide a solution that is able to (1) handle the traffic signal control when certain surveillance cameras that retrieve information for reinforcement learning are down, (2) learn from batch data without a traffic simulator, and (3) make control decisions without shared information across intersections. We present a two-stage framework to deal with the above-mentioned situations. The framework can be decomposed into an Evolution Strategies approach that gives a fixed-time traffic signal control schedule and a multi-agent off-policy reinforcement learning that is capable of learning from batch data with the aid of three proposed components, bounded action, batch augmentation, and surrogate reward clipping. Our experiments show that the proposed framework reduces traffic congestion by 36% in terms of waiting time compared with the currently used fixed-time traffic signal plan. Furthermore, the framework requires only 600 queries to a simulator to achieve the result.

Path and motion planning for autonomous systems has long been an area of research within the robotics and artificial intelligence communities. This has led to the development of a number of frameworks which formulate planning tasks in terms of mathematical optimization problems, which can then be solved by utilizing approaches from optimization theory and optimal control [1, 2]. However, planning in complex domains can be a challenging problem, and requires the agents to spend time and computational resources in order to find solutions, leading to an intrinsic need to balance computational complexity and optimality [3, 4, 5, 6, 7]. Within the path-planning community, this observation has resulted in the development of a number of approaches, which aim to explicitly capture the interplay between complexity and optimality. For example, in [8, 5, 9], the authors utilize wavelets to obtain multi-resolution representations of a two-dimensional environment for path-planning.