Evolutionary Algorithms (EAs) and Deep Reinforcement Learning (DRL) have recently been integrated to take the advantage of the both methods for better exploration and exploitation. The evolutionary part in these hybrid methods maintains a population of policy networks. However, existing methods focus on optimizing the parameters of policy network, which is usually high-dimensional and tricky for EA. In this paper, we shift the target of evolution from high-dimensional parameter space to low-dimensional action space. We propose Evolutionary Action Selection-Twin Delayed Deep Deterministic Policy Gradient (EAS-TD3), a novel hybrid method of EA and DRL. In EAS, we focus on optimizing the action chosen by the policy network and attempt to obtain high-quality actions to promote policy learning through an evolutionary algorithm. We conduct several experiments on challenging continuous control tasks. The result shows that EAS-TD3 shows superior performance over other state-of-art methods.

But in the literature there lacks discussion on how to build the quantitative relationship between them. In this paper, we propose an agent-based SIR model: ASIR. ASIR can robustly reproduce the infection curve predicted by a given SIR model (the simplest CM.) Notably, one can deduce any parameter of ASIR from parameters of SIR without manual tuning. ASIR offers epidemiologists a method to transform a calibrated SIR model into an agent-based model that inherit SIR's performance without another round of calibration. The design ASIR is inspirational for building a general quantitative relationship between CM and AM.

In this paper we study estimating Generalized Linear Models (GLMs) in the case where the agents (individuals) are strategic or self-interested and they concern about their privacy when reporting data. Compared with the classical setting, here we aim to design mechanisms that can both incentivize most agents to truthfully report their data and preserve the privacy of individuals' reports, while their outputs should also close to the underlying parameter. In the first part of the paper, we consider the case where the covariates are sub-Gaussian and the responses are heavy-tailed where they only have the finite fourth moments. First, motivated by the stationary condition of the maximizer of the likelihood function, we derive a novel private and closed form estimator. Based on the estimator, we propose a mechanism which has the following properties via some appropriate design of the computation and payment scheme for several canonical models such as linear regression, logistic regression and Poisson regression: (1) the mechanism is $o(1)$-jointly differentially private (with probability at least $1-o(1)$); (2) it is an $o(\frac{1}{n})$-approximate Bayes Nash equilibrium for a $(1-o(1))$-fraction of agents to truthfully report their data, where $n$ is the number of agents; (3) the output could achieve an error of $o(1)$ to the underlying parameter; (4) it is individually rational for a $(1-o(1))$ fraction of agents in the mechanism ; (5) the payment budget required from the analyst to run the mechanism is $o(1)$. In the second part, we consider the linear regression model under more general setting where both covariates and responses are heavy-tailed and only have finite fourth moments. By using an $\ell_4$-norm shrinkage operator, we propose a private estimator and payment scheme which have similar properties as in the sub-Gaussian case.

The article proposes a universal dual-axis intelligent systems assessment scale. The scale considers the properties of intelligent systems within the environmental context, which develops over time. In contrast to the frequent consideration of the 'mind' of artificial intelligent systems on a scale from 'weak' to 'strong', we highlight the modulating influences of anticipatory ability on their 'brute force'. In addition, the complexity, the 'weight' of the cognitive task and the ability to critically assess it beforehand determine the actual set of cognitive tools, the use of which provides the best result in these conditions. In fact, the presence of 'common sense' options is what connects the ability to solve a problem with the correct use of such an ability itself. The degree of 'correctness' and 'adequacy' is determined by the combination of a suitable solution with the temporal characteristics of the event, phenomenon, object or subject under study.

Reinforcement learning (RL) is an effective method for solving problems that require agents to learn the best way to act in complex environments. RLlib is a powerful tool for applying reinforcement learning to problems where there are multiple agents or when agents must take on multiple roles. There are many of resources for learning about RLlib from a theoretical or academic perspective, but there is a lack of materials for learning how to use RLlib to solve your own practical problems. This tutorial helps to fill that gap. If you want to get right into RLlib, fell free to skip to the next section. Thorndike observed that some behaviors in animals arise from a gradual stamping in [Thorndike, 1898].

In decentralized optimization environments, each agent $i$ in a network of $n$ nodes has its own private function $f_i$, and nodes communicate with their neighbors to cooperatively minimize the aggregate objective $\sum_{i=1}^n f_i$. In this setting, synchronizing the nodes' updates incurs significant communication overhead and computational costs, so much of the recent literature has focused on the analysis and design of asynchronous optimization algorithms, where agents activate and communicate at arbitrary times without needing a global synchronization enforcer. However, most works assume that when a node activates, it selects the neighbor to contact based on a fixed probability (e.g., uniformly at random), a choice that ignores the optimization landscape at the moment of activation. Instead, in this work we introduce an optimization-aware selection rule that chooses the neighbor providing the highest dual cost improvement (a quantity related to a dualization of the problem based on consensus). This scheme is related to the coordinate descent (CD) method with the Gauss-Southwell (GS) rule for coordinate updates; in our setting however, only a subset of coordinates is accessible at each iteration (because each node can communicate only with its neighbors), so the existing literature on GS methods does not apply. To overcome this difficulty, we develop a new analytical framework for smooth and strongly convex $f_i$ that covers the class of set-wise CD algorithms -- a class that directly applies to decentralized scenarios, but is not limited to them -- and we show that the proposed set-wise GS rule achieves a speedup factor of up to the maximum degree in the network (which is in the order of $\Theta(n)$ for highly connected graphs). The speedup predicted by our analysis is validated in numerical experiments with synthetic data.

In this paper, classic controllability and structural controllability under two protocols are investigated. For classic controllability, the multiplicity of eigenvalue zero of general Laplacian matrix $L^*$ is shown to be determined by the sum of the numbers of zero circles, identical nodes and opposite pairs, while it is always simple for the Laplacian $L$ with diagonal entries in absolute form. For a fixed structurally balanced topology, the controllable subspace is proved to be invariant even if the antagonistic weights are selected differently under the corresponding protocol with $L$. For a graph expanded from a star graph rooted from a single leader, the dimension of controllable subspace is two under the protocol associated with $L^*$. In addition, the system is structurally controllable under both protocols if and only if the topology without unaccessible nodes is connected. As a reinforcing case of structural controllability, strong structural controllability requires the system to be controllable for any choice of weights. The connection between father nodes and child nodes affects strong structural controllability because it determines the linear relationship of the control information from father nodes. This discovery is a major factor in establishing the sufficient conditions on strong structural controllability for multi-agent systems under both protocols, rather than for complex networks, about latter results are already abundant.

The diversified ecology in nature had various forms of swarm behaviors in many species. The butterfly species is one of the prominent and a bit insightful in their random flights and converting that into an artificial metaphor would lead to enormous possibilities. This paper considers one such metaphor known as Butterfly Mating Optimization (BMO). In BMO, the Bfly follows the patrolling mating phenomena and simultaneously captures all the local optima of multimodal functions. To imitate this algorithm, a mobile robot (Bflybot) was designed to meet the features of the Bfly in the BMO algorithm. Also, the multi-Bflybot swarm is designed to act like butterflies in nature and follow the algorithm's rules. The real-time experiments were performed on the BMO algorithm in the multi-robotic arena and considered the signal source as the light source. The experimental results show that the BMO algorithm is applicable to detect multiple signal sources with significant variations in their movements i.e., static and dynamic. In the case of static signal sources, with varying initial locations of Bflybots, the convergence is affected in terms of time and smoothness. Whereas the experiments with varying step-size leads to their variation in the execution time and speed of the bots. In this work, experiments were performed in a dynamic environment where the movement of the signal source in both maneuvering and non-maneuvering scenarios. The Bflybot swarm is able to detect the single and multi-signal sources, moving linearly in between two fixed points, in circular, up and down movements.To evaluate the BMO phenomenon, various ongoing and prospective works such as mid-sea ship detection, aerial search applications, and earthquake prediction were discussed.

Lying on the heart of intelligent decision-making systems, how policy is represented and optimized is a fundamental problem. The root challenge in this problem is the large scale and the high complexity of policy space, which exacerbates the difficulty of policy learning especially in real-world scenarios. Towards a desirable surrogate policy space, recently policy representation in a low-dimensional latent space has shown its potential in improving both the evaluation and optimization of policy. The key question involved in these studies is by what criterion we should abstract the policy space for desired compression and generalization. However, both the theory on policy abstraction and the methodology on policy representation learning are less studied in the literature. In this work, we make very first efforts to fill up the vacancy. First, we propose a unified policy abstraction theory, containing three types of policy abstraction associated to policy features at different levels. Then, we generalize them to three policy metrics that quantify the distance (i.e., similarity) of policies, for more convenient use in learning policy representation. Further, we propose a policy representation learning approach based on deep metric learning. For the empirical study, we investigate the efficacy of the proposed policy metrics and representations, in characterizing policy difference and conveying policy generalization respectively. Our experiments are conducted in both policy optimization and evaluation problems, containing trust-region policy optimization (TRPO), diversity-guided evolution strategy (DGES) and off-policy evaluation (OPE). Somewhat naturally, the experimental results indicate that there is no a universally optimal abstraction for all downstream learning problems; while the influence-irrelevance policy abstraction can be a generally preferred choice.

In natural language, referencing objects at different levels of specificity is a fundamental pragmatic mechanism for efficient communication in context. We develop a novel communication game, the hierarchical reference game, to study the emergence of such reference systems in artificial agents. We consider a simplified world, in which concepts are abstractions over a set of primitive attributes (e.g., color, style, shape). Depending on how many attributes are combined, concepts are more general ("circle") or more specific ("red dotted circle"). Based on the context, the agents have to communicate at different levels of this hierarchy. Our results show that the agents learn to play the game successfully and can even generalize to novel concepts. To achieve abstraction, they use implicit (omitting irrelevant information) and explicit (indicating that attributes are irrelevant) strategies. In addition, the compositional structure underlying the concept hierarchy is reflected in the emergent protocols, indicating that the need to develop hierarchical reference systems supports the emergence of compositionality.