We consider a repeated sequential game between a learner, who plays first, and an opponent who responds to the chosen action. We seek to design strategies for the learner to successfully interact with the opponent. While most previous approaches consider known opponent models, we focus on the setting in which the opponent's model is unknown. To this end, we use kernel-based regularity assumptions to capture and exploit the structure in the opponent's response. We propose a novel algorithm for the learner when playing against an adversarial sequence of opponents. The algorithm combines ideas from bilevel optimization and online learning to effectively balance between exploration (learning about the opponent's model) and exploitation (selecting highly rewarding actions for the learner). Our results include algorithm's regret guarantees that depend on the regularity of the opponent's response and scale sublinearly with the number of game rounds. Moreover, we specialize our approach to repeated Stackelberg games, and empirically demonstrate its effectiveness in a traffic routing and wildlife conservation task

Analyzing encircling situation is the most crucial part of autonomous adaptation. Since there are many unknown and constantly changing factors in the real environment, momentary adjustment to the consistently alternating circumstances is highly required for addressing autonomy. To respond properly to changing environment, an utterly self-ruling vehicle ought to have the capacity to realize/comprehend its particular position and the surrounding environment. However, these vehicles extremely rely on human involvement to resolve entangled missions that cannot be precisely characterized in advance, which restricts their applications and accuracy. Reducing dependence on human supervision can be achieved by improving level of autonomy. Over the previous decades, autonomy and mission planning have been extensively researched on different structures and diverse conditions; nevertheless, aiming at robust mission planning in extreme conditions, here we provide exhaustive study of UVs autonomy as well as its related properties in internal and external situation awareness. In the following discussion, different difficulties in the scope of AUVs and UAVs will be discussed.

We train an agent to navigate in 3D environments using a hierarchical strategy including a high-level graph based planner and a local policy. Our main contribution is a data driven learning based approach for planning under uncertainty in topological maps, requiring an estimate of shortest paths in valued graphs with a probabilistic structure. Whereas classical symbolic algorithms achieve optimal results on noise-less topologies, or optimal results in a probabilistic sense on graphs with probabilistic structure, we aim to show that machine learning can overcome missing information in the graph by taking into account rich high-dimensional node features, for instance visual information available at each location of the map. Compared to purely learned neural white box algorithms, we structure our neural model with an inductive bias for dynamic programming based shortest path algorithms, and we show that a particular parameterization of our neural model corresponds to the Bellman-Ford algorithm. By performing an empirical analysis of our method in simulated photo-realistic 3D environments, we demonstrate that the inclusion of visual features in the learned neural planner outperforms classical symbolic solutions for graph based planning.

Usage of automated controllers which make decisions on an environment are widespread and are often based on black-box models. We use Knowledge Compilation theory to bring explainability to the controller's decision given the state of the system. For this, we use simulated historical state-action data as input and build a compact and structured representation which relates states with actions. We implement this method in a Traffic Light Control scenario where the controller selects the light cycle by observing the presence (or absence) of vehicles in different regions of the incoming roads.

Autonomous agents must learn to collaborate. It is not scalable to develop a new centralized agent every time a task's difficulty outpaces a single agent's abilities. While multi-agent collaboration research has flourished in gridworld-like environments, relatively little work has considered visually rich domains. Addressing this, we introduce the novel task FurnMove in which agents work together to move a piece of furniture through a living room to a goal. Unlike existing tasks, FurnMove requires agents to coordinate at every timestep. We identify two challenges when training agents to complete FurnMove: existing decentralized action sampling procedures do not permit expressive joint action policies and, in tasks requiring close coordination, the number of failed actions dominates successful actions. To confront these challenges we introduce SYNC-policies (synchronize your actions coherently) and CORDIAL (coordination loss). Using SYNC-policies and CORDIAL, our agents achieve a 58% completion rate on FurnMove, an impressive absolute gain of 25 percentage points over competitive decentralized baselines. Our dataset, code, and pretrained models are available at https://unnat.github.io/cordial-sync .

Many real-world situations of ethical relevance, in particular those of large-scale social choice such as mitigating climate change, involve not only many agents whose decisions interact in complicated ways, but also various forms of uncertainty, including quantifiable risk and unquantifiable ambiguity. In such problems, an assessment of individual and groupwise moral responsibility for ethically undesired outcomes or their responsibility to avoid such is challenging and prone to the risk of under- or overdetermination of responsibility. In contrast to existing approaches based on strict causation or certain deontic logics that focus on a binary classification of `responsible' vs `not responsible', we here present several different quantitative responsibility metrics that assess responsibility degrees in units of probability. For this, we use a framework based on an adapted version of extensive-form game trees and an axiomatic approach that specifies a number of potentially desirable properties of such metrics, and then test the developed candidate metrics by their application to a number of paradigmatic social choice situations. We find that while most properties one might desire of such responsibility metrics can be fulfilled by some variant, an optimal metric that clearly outperforms others has yet to be found.

Smart FinTech has emerged as a new area that synthesizes and transforms AI and finance, and broadly data science, machine learning, economics, etc. Smart FinTech also transforms and drives new economic and financial businesses, services and systems, and plays an increasingly important role in economy, technology and society transformation. This article presents a highly summarized research overview of smart FinTech, including FinTech businesses and challenges, various FinTech-associated data and repositories, FinTech-driven business decision and optimization, areas in smart FinTech, and research methods and techniques for smart FinTech.

In recent years, simulation techniques have been applied to investigate the spatiotemporal dynamics of crime. Researchers have instantiated mobile offenders in agent-based simulations for theory testing, experimenting with crime prevention strategies, and exploring crime prediction techniques, despite facing challenges due to the complex dynamics of crime and the lack of detailed information about offender mobility. This paper presents a simulation model to explore offender mobility, focusing on the interplay between the agent's awareness space and activity nodes. The simulation generates patterns of individual mobility aiming to cumulatively match crime patterns. To instantiate a realistic urban environment, we use open data to simulate the urban structure, location-based social networks data to represent activity nodes as a proxy for human activity, and taxi trip data as a proxy for human movement between regions of the city. We analyze and systematically compare 35 different mobility strategies and demonstrate the benefits of using large-scale human activity data to simulate offender mobility. The strategies combining taxi trip data or historic crime data with popular activity nodes perform best compared to other strategies, especially for robbery. Our approach provides a basis for building agent-based crime simulations that infer offender mobility in urban areas from real-world data.

Reinforcement learning is typically concerned with learning control policies tailored to a particular agent. We investigate whether there exists a single global policy that can generalize to control a wide variety of agent morphologies -- ones in which even dimensionality of state and action spaces changes. We propose to express this global policy as a collection of identical modular neural networks, dubbed as Shared Modular Policies (SMP), that correspond to each of the agent's actuators. Every module is only responsible for controlling its corresponding actuator and receives information from only its local sensors. In addition, messages are passed between modules, propagating information between distant modules. We show that a single modular policy can successfully generate locomotion behaviors for several planar agents with different skeletal structures such as monopod hoppers, quadrupeds, bipeds, and generalize to variants not seen during training -- a process that would normally require training and manual hyperparameter tuning for each morphology. We observe that a wide variety of drastically diverse locomotion styles across morphologies as well as centralized coordination emerges via message passing between decentralized modules purely from the reinforcement learning objective. Videos and code at https://huangwl18.github.io/modular-rl/

The COVID-19 pandemic due to the SARS-CoV-2 coronavirus has directly impacted the public health and economy worldwide. To overcome this problem, countries have adopted different policies and non-pharmaceutical interventions for controlling the spread of the virus. This paper proposes the COVID-ABS, a new SEIR (Susceptible-Exposed-Infected-Recovered) agent-based model that aims to simulate the pandemic dynamics using a society of agents emulating people, business and government. Seven different scenarios of social distancing interventions were analyzed, with varying epidemiological and economic effects: (1) do nothing, (2) lockdown, (3) conditional lockdown, (4) vertical isolation, (5) partial isolation, (6) use of face masks, and (7) use of face masks together with 50% of adhesion to social isolation. In the impossibility of implementing scenarios with lockdown, which present the lowest number of deaths and highest impact on the economy, scenarios combining the use of face masks and partial isolation can be the more realistic for implementation in terms of social cooperation. The COVID-ABS model was implemented in Python programming language, with source code publicly available. The model can be easily extended to other societies by changing the input parameters, as well as allowing the creation of a multitude of other scenarios. Therefore, it is a useful tool to assist politicians and health authorities to plan their actions against the COVID-19 epidemic.