In this work we give a case study of a modular embodied machine-learning (ML) powered agent that improves itself via interactions with crowd-workers. The agent consists of a set of modules, some of which are learned, and others heuristic. While the agent is not "end-to-end" in the ML sense, end-to-end interaction with humans and its environment is a vital part of the agent's learning mechanism. We describe how the design of the agent works together with the design of multiple annotation interfaces to allow crowd-workers to assign credit to module errors from these end-toend interactions, and to label data for an individual module. We further show how this whole loop (including model re-training and re-deployment) can be automated. Over multiple loops with crowdsourced humans with no knowledge of the agent architecture, we demonstrate improvement over the agent's language understanding and visual perception modules. Present day machine learning (ML) research prioritizes end-to-end learning. Not only are end-to-end models able to achieve excellent performance on static tasks, there is a growing literature on how to adapt pre-trained networks to new tasks, and large pre-trained models can have impressive zero-shot performance on unseen tasks. In the setting of embodied agents, this manifests as agents actualized as monolithic ML models, where inputs to the model are the agent's perceptual sensors, and the model's outputs directly control agent actions. There are now a number of environments designed for the training of end-to-end embodied agents Beattie et al. (2016); Savva et al. (2019); Guss et al. (2019); Petrenko et al. (2021), and there is hope (and some evidence) that the same sort of transfer and adaptability seen in language and vision models will carry over to the embodied agent setting. Nevertheless, agents implemented as fully end-to-end ML models are rare in production systems (or in real-world embodied agents, a.k.a. While this in part is a symptom of the rapid improvement and scaling in the literature and the lag in technology transfer, these systems require performance and safety guarantees that are still not easily obtainable from end-to-end ML models; and must be maintainable by human engineers.

We introduce a novel framework for causal explanations of stochastic, sequential decision-making systems built on the well-studied structural causal model paradigm for causal reasoning. This single framework can identify multiple, semantically distinct explanations for agent actions -- something not previously possible. In this paper, we establish exact methods and several approximation techniques for causal inference on Markov decision processes using this framework, followed by results on the applicability of the exact methods and some run time bounds. We discuss several scenarios that illustrate the framework's flexibility and the results of experiments with human subjects that confirm the benefits of this approach.

Abstract--Understanding traffic scenes requires considering heterogeneous information about dynamic agents and the static infrastructure. Task-specific decoders can be applied to predict desired attributes of the scene. To this end, the vehicle needs to correctly estimate which sensory information is reliable I. NDERSTANDING traffic scenes is important for an autonomous vehicle such that it may develop a safe, agents is conveyed by the perception systems of autonomous effective and efficient plan of how to move forward. We raise the hypothesis that considering additional instance, whether a stationary car is parked or just temporarily heterogeneous entities in a traffic scene might add valuable stopped determines whether the autonomous vehicle should information. In particular, reasoning should also involve wait or overtake. Understanding of traffic scenes requires knowledge about static infrastructure, which may either be reasoning about dynamic agents and static infrastructure in perceived or in our case is provided by a High Definition order to predict the intents of nearby dynamic agents (e.g., (HD) map.

A two-player game-theoretic problem on resilient graphs in a multiagent consensus setting is formulated. An attacker is capable to disable some of the edges of the network with the objective to divide the agents into clusters by emitting jamming signals while, in response, the defender recovers some of the edges by increasing the transmission power for the communication signals. Specifically, we consider repeated games between the attacker and the defender where the optimal strategies for the two players are derived in a rolling horizon fashion based on utility functions that take both the agents' states and the sizes of clusters (known as network effect) into account. The players' actions at each discrete-time step are constrained by their energy for transmissions of the signals, with a less strict constraint for the attacker. Necessary conditions and sufficient conditions of agent consensus are derived, which are influenced by the energy constraints. The number of clusters of agents at infinite time in the face of attacks and recoveries are also characterized. Simulation results are provided to demonstrate the effects of players' actions on the cluster forming and to illustrate the players' performance for different horizon parameters.

The notion of safety in multi-agent systems assumes great significance in many emerging collaborative multi-robot applications. In this paper, we present a multi-UAV collaborative target-tracking application by defining bounded inter-UAV distances in the formation in order to ensure safe operation. In doing so, we address the problem of prioritizing specific objectives over others in a multi-objective control framework. We propose a barrier Lyapunov function-based distributed control law to enforce the bounds on the distances and assess its Lyapunov stability using a kinematic model. The theoretical analysis is supported by numerical results, which account for measurement noise and moving targets. Straight-line and circular motion of the target are considered, and results for quadratic Lyapunov function-based control, often used in multi-agent multi-objective problems, are also presented. A comparison of the two control approaches elucidates the advantages of our proposed safe-control in bounding the inter-agent distances in a formation. A concluding evaluation using ROS simulations illustrates the practical applicability of the proposed control to a pair of multi-rotors visually estimating and maintaining their mutual separation within specified bounds, as they track a moving target.

In the DRL case, mimic learning aims to replace the closedbox successfully applied in challenging tasks, their application in realworld DRL controller with an interpretable one, able to mimic the operational settings is challenged by methods' limited ability decisions made by the former [3, 19, 35]. A mimic learner tries to to provide explanations. Among the paradigms for explainability in optimize fidelity [35], which is determined by comparing the mimic DRL is the interpretable box design paradigm, where interpretable controller's actions with the actions selected by the DRL model. To models substitute inner constituent models of the DRL method, thus extract knowledge from deep neural networks, recent work [3, 19] making the DRL method "inherently" interpretable. In this paper has applied mimic learning with tree representations, using decision we explore this paradigm and we propose XDQN, an explainable trees: Criteria used for splitting tree nodes provide a tractable way variation of DQN, which uses an interpretable policy model trained to explain the predictions made by the controller.

The players can cooperate to achieve an objective, and compete against other players with conflicting interests. Creating artificial intelligence agents that can learn to compete and cooperate as effectively as humans remains a thorny problem. A key challenge is enabling AI agents to anticipate future behaviors of other agents when they are all learning simultaneously. Because of the complexity of this problem, current approaches tend to be myopic; the agents can only guess the next few moves of their teammates or competitors, which leads to poor performance in the long run. Researchers from MIT, the MIT-IBM Watson AI Lab, and elsewhere have developed a new approach that gives AI agents a farsighted perspective.

Captioning is a crucial and challenging task for video understanding. In videos that involve active agents such as humans, the agent's actions can bring about myriad changes in the scene. Observable changes such as movements, manipulations, and transformations of the objects in the scene, are reflected in conventional video captioning. Unlike images, actions in videos are also inherently linked to social aspects such as intentions (why the action is taking place), effects (what changes due to the action), and attributes that describe the agent. Thus for video understanding, such as when captioning videos or when answering questions about videos, one must have an understanding of these commonsense aspects. We present the first work on generating commonsense captions directly from videos, to describe latent aspects such as intentions, effects, and attributes. We present a new dataset "Video-to-Commonsense (V2C)" that contains $\sim9k$ videos of human agents performing various actions, annotated with 3 types of commonsense descriptions. Additionally we explore the use of open-ended video-based commonsense question answering (V2C-QA) as a way to enrich our captions. Both the generation task and the QA task can be used to enrich video captions.

We present a non-asymptotic lower bound on the eigenspectrum of the design matrix generated by any linear bandit algorithm with sub-linear regret when the action set has well-behaved curvature. Specifically, we show that the minimum eigenvalue of the expected design matrix grows as $\Omega(\sqrt{n})$ whenever the expected cumulative regret of the algorithm is $O(\sqrt{n})$, where $n$ is the learning horizon, and the action-space has a constant Hessian around the optimal arm. This shows that such action-spaces force a polynomial lower bound rather than a logarithmic lower bound, as shown by \cite{lattimore2017end}, in discrete (i.e., well-separated) action spaces. Furthermore, while the previous result is shown to hold only in the asymptotic regime (as $n \to \infty$), our result for these "locally rich" action spaces is any-time. Additionally, under a mild technical assumption, we obtain a similar lower bound on the minimum eigen value holding with high probability. We apply our result to two practical scenarios -- \emph{model selection} and \emph{clustering} in linear bandits. For model selection, we show that an epoch-based linear bandit algorithm adapts to the true model complexity at a rate exponential in the number of epochs, by virtue of our novel spectral bound. For clustering, we consider a multi agent framework where we show, by leveraging the spectral result, that no forced exploration is necessary -- the agents can run a linear bandit algorithm and estimate their underlying parameters at once, and hence incur a low regret.

The fireworks algorithm is an optimization algorithm for simulating the explosion phenomenon of fireworks. Because of its fast convergence and high precision, it is widely used in pattern recognition, optimal scheduling, and other fields. However, most of the existing research work on the fireworks algorithm is improved based on its defects, and little consideration is given to reducing the number of parameters of the fireworks algorithm. The original fireworks algorithm has too many parameters, which increases the cost of algorithm adjustment and is not conducive to engineering applications. In addition, in the fireworks population, the unselected individuals are discarded, thus causing a waste of their location information. To reduce the number of parameters of the original Fireworks Algorithm and make full use of the location information of discarded individuals, we propose a simplified version of the Fireworks Algorithm. It reduces the number of algorithm parameters by redesigning the explosion operator of the fireworks algorithm and constructs an adaptive explosion radius by using the historical optimal information to balance the local mining and global exploration capabilities. The comparative experimental results of function optimization show that the overall performance of our proposed LFWA is better than that of comparative algorithms, such as the fireworks algorithm, particle swarm algorithm, and bat algorithm.