Reinforcement learning (RL) is an effective approach to learn an optimal dialog policy for task-oriented visual dialog systems. A common practice is to apply RL on a neural sequence-to-sequence (seq2seq) framework with the action space being the output vocabulary in the decoder. However, it is difficult to design a reward function that can achieve a balance between learning an effective policy and generating a natural dialog response. This paper proposes a novel framework that alternatively trains a RL policy for image guessing and a supervised seq2seq model to improve dialog generation quality. We evaluate our framework on the GuessWhich task and the framework achieves the state-of-the-art performance in both task completion and dialog quality.

This paper focuses on the task of generating long structured sentences with explicit discourse markers, by proposing a new task Sentence Transfer and a novel model architecture TransSent. Previous works on text generation fused semantic and structure information in one mixed hidden representation. However, the structure was difficult to maintain properly when the generated sentence became longer. In this work, we explicitly separate the modeling process of semantic information and structure information. Intuitively, humans produce long sentences by directly connecting discourses with discourse markers like and, but, etc. We thus define a new task called Sentence Transfer. This task represents a long sentence as (head discourse, discourse marker, tail discourse) and aims at tail discourse generation based on head discourse and discourse marker. Then, by connecting original head discourse and generated tail discourse with a discourse marker, we generate a long structured sentence. We also propose a model architecture called TransSent, which models relations between two discourses by interpreting them as transferring from one discourse to the other in the embedding space. Experiment results show that our model achieves better performance in automatic evaluations, and can generate structured sentences with high quality. The datasets can be accessed by https://github.com/1024er/TransSent dataset.

Robustness of Deep Reinforcement Learning (DRL) algorithms towards adversarial attacks in real world applications such as those deployed in cyber-physical systems (CPS) are of increasing concern. Numerous studies have investigated the mechanisms of attacks on the RL agent's state space. Nonetheless, attacks on the RL agent's action space (AS) (corresponding to actuators in engineering systems) are equally perverse; such attacks are relatively less studied in the ML literature. In this work, we first frame the problem as an optimization problem of minimizing the cumulative reward of an RL agent with decoupled constraints as the budget of attack. We propose a white-box Myopic Action Space (MAS) attack algorithm that distributes the attacks across the action space dimensions. Next, we reformulate the optimization problem above with the same objective function, but with a temporally coupled constraint on the attack budget to take into account the approximated dynamics of the agent. This leads to the white-box Look-ahead Action Space (LAS) attack algorithm that distributes the attacks across the action and temporal dimensions. Our results shows that using the same amount of resources, the LAS attack deteriorates the agent's performance significantly more than the MAS attack. This reveals the possibility that with limited resource, an adversary can utilize the agent's dynamics to malevolently craft attacks that causes the agent to fail. Additionally, we leverage these attack strategies as a possible tool to gain insights on the potential vulnerabilities of DRL agents.

In many domains, including healthcare and e-commerce, we frequently encounter the following decision-making problem: we sequentially and repeatedly receive context information X (e.g., features of patients or users), need to choose an action A { 1, 1} (e.g., whether to treat a patient with invasive therapy or whether expose a user to our ad), and receive a reward Y ( A) (e.g., patient's health outcome or user's click minus ad spot costs) corresponding to the chosen action. Our goal is to collect the most reward over time. When contexts X and potential rewards Y ( 1),Y ( 1) are drawn from a stationary, but unknown, distribution, this setting is modeled by the stochastic bandit problem (Bubeck and Cesa-Bianchi, 2012, Wang et al., 2005). A special case is the multi-armed bandit (MAB) problem where there is no contextual information (Auer et al., 2002, Lai and Robbins, 1985). In these problems, we quantify the quality of an algorithm for choosing actions based on available historical data in terms of its regret for every horizon T: the expected additional cumulative reward up to time T that we would obtain if we had full knowledge of the stationary context-reward distribution (but not the realizations). The minimax regret is the best (over algorithms) worst-case regret (over problem instances).

Trajectory modelling had been the principal research area for understanding and anticipating human behaviour. Predicting the dynamic path by observing the agent and its surrounding environment are essential for applications such as autonomous driving and indoor navigation suggestions. However, despite the numerous researches that had been presented, most available dataset does not contains any information on environmental factors---such as the occupancy representation of the map---which arguably plays a significant role on how an agent chooses its trajectory. We present a trajectory dataset with the corresponding occupancy representations of different local-maps. The dataset contains more than 120 locally-structured maps with occupancy representation and more than 110K trajectories in total. Each map has few hundred corresponding simulated trajectories that navigate from a spatial location of a room to another point. The dataset is freely available online.

The rapid advancement of artificial intelligence (AI) is changing our lives in many ways. One application domain is data science. New techniques in automating the creation of AI, known as AutoAI or AutoML, aim to automate the work practices of data scientists. AutoAI systems are capable of autonomously ingesting and pre-processing data, engineering new features, and creating and scoring models based on a target objectives (e.g. accuracy or run-time efficiency). Though not yet widely adopted, we are interested in understanding how AutoAI will impact the practice of data science. We conducted interviews with 20 data scientists who work at a large, multinational technology company and practice data science in various business settings. Our goal is to understand their current work practices and how these practices might change with AutoAI. Reactions were mixed: while informants expressed concerns about the trend of automating their jobs, they also strongly felt it was inevitable. Despite these concerns, they remained optimistic about their future job security due to a view that the future of data science work will be a collaboration between humans and AI systems, in which both automation and human expertise are indispensable.

A grouping problem involves partitioning a set of items into mutually disjoint groups or clusters according to some guiding decision criteria and imperative constraints. Grouping problems have many relevant applications and are computationally difficult. In this work, we present a general weight learning based optimization framework for solving grouping problems. The central idea of our approach is to formulate the task of seeking a solution as a real-valued weight matrix learning problem that is solved by first order gradient descent. A practical implementation of this framework is proposed with tensor calculus in order to benefit from parallel computing on GPU devices. To show its potential for tackling difficult problems, we apply the approach to two typical and well-known grouping problems (graph coloring and equitable graph coloring). We present large computational experiments and comparisons on popular benchmarks and report improved best-known results (new upper bounds) for several large graphs.

For more on Apache Kafka, Apache Pulsar, Apache Spark, and other data technologies, attend the "Data Engineering & Architecture" sessions at the Strata Data Conference in New York City, September 23-26, 2019. With companies producing data from an increasing number of systems and devices, messaging and event streaming solutions--particularly Apache Kafka--have gained widespread adoption. Over the past year, we've been tracking the progress of Apache Pulsar (Pulsar), a less well-known but highly capable open source solution originated by Yahoo. Pulsar is designed to intelligently process, analyze, and deliver data from an expanding array of services and applications, and thus it fits nicely into modern data platforms. Pulsar is also designed to ease the operational burdens normally associated with complex, distributed systems.

Industry 4.0 - the fourth industrial revolution – is upon us. Artificial intelligence (AI) is forever changing the way information is used across all business lines of the Government and private sector alike. An important DoD priority is to use AI to improve maintenance activities. However, AI depends on the quality of data, so the DoD must first be able to capture data on maintenance activities that is complete, structured, and readily accessible. DoD maintenance faces growing challenges that threaten the strategic advantage the United States Military has long held in both combat and humanitarian missions.

On Monday, September 2, 2019, the summer school "IoT meets AI" started very successfully in Werk 1 in Munich. The team around Marc-Oliver Pahl had invested a significant portion of their weekend on setting up a cozy work environment on the one hand, and preparing everything for the livestream (https://future-iot.org/stream/) on the other hand. Marc-Oliver Pahl (TUM) opened the summer school by introducing his co-organizers Arne Bröring (Siemens) and Nicolas Montavont (IMT). He credited the organizing team that did a wonderful job in helping to setup the event for the 32 participants and about 30 support staff people. In particular he mentioned Fabian Rhein (Siemens), Catharina Vos (TUM International), his team consisting of Lars Wüstrich, Erkin Kirdan, Stefan Liebald, and Christian Lübben, and Olivia Pahl (FGW), who helped almost the entire weekend, and Marlene Eder (Werk1).