The article proposes a universal dual-axis intelligent systems assessment scale. The scale considers the properties of intelligent systems within the environmental context, which develops over time. In contrast to the frequent consideration of the 'mind' of artificial intelligent systems on a scale from 'weak' to 'strong', we highlight the modulating influences of anticipatory ability on their 'brute force'. In addition, the complexity, the 'weight' of the cognitive task and the ability to critically assess it beforehand determine the actual set of cognitive tools, the use of which provides the best result in these conditions. In fact, the presence of 'common sense' options is what connects the ability to solve a problem with the correct use of such an ability itself. The degree of 'correctness' and 'adequacy' is determined by the combination of a suitable solution with the temporal characteristics of the event, phenomenon, object or subject under study.

Reinforcement learning (RL) is an effective method for solving problems that require agents to learn the best way to act in complex environments. RLlib is a powerful tool for applying reinforcement learning to problems where there are multiple agents or when agents must take on multiple roles. There are many of resources for learning about RLlib from a theoretical or academic perspective, but there is a lack of materials for learning how to use RLlib to solve your own practical problems. This tutorial helps to fill that gap. If you want to get right into RLlib, fell free to skip to the next section. Thorndike observed that some behaviors in animals arise from a gradual stamping in [Thorndike, 1898].

In decentralized optimization environments, each agent $i$ in a network of $n$ nodes has its own private function $f_i$, and nodes communicate with their neighbors to cooperatively minimize the aggregate objective $\sum_{i=1}^n f_i$. In this setting, synchronizing the nodes' updates incurs significant communication overhead and computational costs, so much of the recent literature has focused on the analysis and design of asynchronous optimization algorithms, where agents activate and communicate at arbitrary times without needing a global synchronization enforcer. However, most works assume that when a node activates, it selects the neighbor to contact based on a fixed probability (e.g., uniformly at random), a choice that ignores the optimization landscape at the moment of activation. Instead, in this work we introduce an optimization-aware selection rule that chooses the neighbor providing the highest dual cost improvement (a quantity related to a dualization of the problem based on consensus). This scheme is related to the coordinate descent (CD) method with the Gauss-Southwell (GS) rule for coordinate updates; in our setting however, only a subset of coordinates is accessible at each iteration (because each node can communicate only with its neighbors), so the existing literature on GS methods does not apply. To overcome this difficulty, we develop a new analytical framework for smooth and strongly convex $f_i$ that covers the class of set-wise CD algorithms -- a class that directly applies to decentralized scenarios, but is not limited to them -- and we show that the proposed set-wise GS rule achieves a speedup factor of up to the maximum degree in the network (which is in the order of $\Theta(n)$ for highly connected graphs). The speedup predicted by our analysis is validated in numerical experiments with synthetic data.

In this paper, classic controllability and structural controllability under two protocols are investigated. For classic controllability, the multiplicity of eigenvalue zero of general Laplacian matrix $L^*$ is shown to be determined by the sum of the numbers of zero circles, identical nodes and opposite pairs, while it is always simple for the Laplacian $L$ with diagonal entries in absolute form. For a fixed structurally balanced topology, the controllable subspace is proved to be invariant even if the antagonistic weights are selected differently under the corresponding protocol with $L$. For a graph expanded from a star graph rooted from a single leader, the dimension of controllable subspace is two under the protocol associated with $L^*$. In addition, the system is structurally controllable under both protocols if and only if the topology without unaccessible nodes is connected. As a reinforcing case of structural controllability, strong structural controllability requires the system to be controllable for any choice of weights. The connection between father nodes and child nodes affects strong structural controllability because it determines the linear relationship of the control information from father nodes. This discovery is a major factor in establishing the sufficient conditions on strong structural controllability for multi-agent systems under both protocols, rather than for complex networks, about latter results are already abundant.

The diversified ecology in nature had various forms of swarm behaviors in many species. The butterfly species is one of the prominent and a bit insightful in their random flights and converting that into an artificial metaphor would lead to enormous possibilities. This paper considers one such metaphor known as Butterfly Mating Optimization (BMO). In BMO, the Bfly follows the patrolling mating phenomena and simultaneously captures all the local optima of multimodal functions. To imitate this algorithm, a mobile robot (Bflybot) was designed to meet the features of the Bfly in the BMO algorithm. Also, the multi-Bflybot swarm is designed to act like butterflies in nature and follow the algorithm's rules. The real-time experiments were performed on the BMO algorithm in the multi-robotic arena and considered the signal source as the light source. The experimental results show that the BMO algorithm is applicable to detect multiple signal sources with significant variations in their movements i.e., static and dynamic. In the case of static signal sources, with varying initial locations of Bflybots, the convergence is affected in terms of time and smoothness. Whereas the experiments with varying step-size leads to their variation in the execution time and speed of the bots. In this work, experiments were performed in a dynamic environment where the movement of the signal source in both maneuvering and non-maneuvering scenarios. The Bflybot swarm is able to detect the single and multi-signal sources, moving linearly in between two fixed points, in circular, up and down movements.To evaluate the BMO phenomenon, various ongoing and prospective works such as mid-sea ship detection, aerial search applications, and earthquake prediction were discussed.

Lying on the heart of intelligent decision-making systems, how policy is represented and optimized is a fundamental problem. The root challenge in this problem is the large scale and the high complexity of policy space, which exacerbates the difficulty of policy learning especially in real-world scenarios. Towards a desirable surrogate policy space, recently policy representation in a low-dimensional latent space has shown its potential in improving both the evaluation and optimization of policy. The key question involved in these studies is by what criterion we should abstract the policy space for desired compression and generalization. However, both the theory on policy abstraction and the methodology on policy representation learning are less studied in the literature. In this work, we make very first efforts to fill up the vacancy. First, we propose a unified policy abstraction theory, containing three types of policy abstraction associated to policy features at different levels. Then, we generalize them to three policy metrics that quantify the distance (i.e., similarity) of policies, for more convenient use in learning policy representation. Further, we propose a policy representation learning approach based on deep metric learning. For the empirical study, we investigate the efficacy of the proposed policy metrics and representations, in characterizing policy difference and conveying policy generalization respectively. Our experiments are conducted in both policy optimization and evaluation problems, containing trust-region policy optimization (TRPO), diversity-guided evolution strategy (DGES) and off-policy evaluation (OPE). Somewhat naturally, the experimental results indicate that there is no a universally optimal abstraction for all downstream learning problems; while the influence-irrelevance policy abstraction can be a generally preferred choice.

In natural language, referencing objects at different levels of specificity is a fundamental pragmatic mechanism for efficient communication in context. We develop a novel communication game, the hierarchical reference game, to study the emergence of such reference systems in artificial agents. We consider a simplified world, in which concepts are abstractions over a set of primitive attributes (e.g., color, style, shape). Depending on how many attributes are combined, concepts are more general ("circle") or more specific ("red dotted circle"). Based on the context, the agents have to communicate at different levels of this hierarchy. Our results show that the agents learn to play the game successfully and can even generalize to novel concepts. To achieve abstraction, they use implicit (omitting irrelevant information) and explicit (indicating that attributes are irrelevant) strategies. In addition, the compositional structure underlying the concept hierarchy is reflected in the emergent protocols, indicating that the need to develop hierarchical reference systems supports the emergence of compositionality.

We argue that intelligence -- construed as the disposition to perform tasks successfully--is a property of systems composed of agents and their contexts. This is the thesis of extended intelligence. We argue that the performance of an agent will generally not be preserved if its context is allowed to vary. Hence, this disposition is not possessed by an agent alone, but is rather possessed by the system consisting of an agent and its context, which we dub an agent-in-context. An agent's context may include an environment, other agents, cultural artifacts (like language, technology), or all of these, as is typically the case for humans and artificial intelligence systems, as well as many non-human animals. In virtue of the thesis of extended intelligence, we contend that intelligence is context-bound, taskparticular and incommensurable among agents. Our thesis carries strong implications for how intelligence is analyzed in the context of both psychology and artificial intelligence.

The use of game theoretic methods for control in multiagent systems has been an important topic in recent research. Valid utility games in particular have been used to model real-world problems; such games have the convenient property that the value of any decision set which is a Nash equilibrium of the game is guaranteed to be within 1/2 of the value of the optimal decision set. However, an implicit assumption in this guarantee is that each agent is aware of the decisions of all other agents. In this work, we first describe how this guarantee degrades as agents are only aware of a subset of the decisions of other agents. We then show that this loss can be mitigated by restriction to a relevant subclass of games.

Adaptive Multi-Agent Systems (AMAS) transform dynamic problems into problems of local cooperation between agents. We present smapy, an ensemble based AMAS implementation for mobility prediction, whose agents are provided with machine learning models in addition to their cooperation rules. With a detailed methodology, we propose a framework to transform a classification problem into a cooperative tiling of the input variable space. We show that it is possible to use linear classifiers for online non-linear classification on three benchmark toy problems chosen for their different levels of linear separability, if they are integrated in a cooperative Multi-Agent structure. The results obtained show a significant improvement of the performance of linear classifiers in non-linear contexts in terms of classification accuracy and decision boundaries, thanks to the cooperative approach.