Distributed fuzzy neural networks (DFNNs) have attracted increasing attention recently due to their learning abilities in handling data uncertainties in distributed scenarios. However, it is challenging for DFNNs to handle cases in which the local data are non-independent and identically distributed (non-IID). In this paper, we propose a federated fuzzy neural network (FedFNN) with evolutionary rule learning (ERL) to cope with non-IID issues as well as data uncertainties. The FedFNN maintains a global set of rules in a server and a personalized subset of these rules for each local client. ERL is inspired by the theory of biological evolution; it encourages rule variations while activating superior rules and deactivating inferior rules for local clients with non-IID data. Specifically, ERL consists of two stages in an iterative procedure: a rule cooperation stage that updates global rules by aggregating local rules based on their activation statuses and a rule evolution stage that evolves the global rules and updates the activation statuses of the local rules. This procedure improves both the generalization and personalization of the FedFNN for dealing with non-IID issues and data uncertainties. Extensive experiments conducted on a range of datasets demonstrate the superiority of the FedFNN over state-of-the-art methods.

Originally published on Towards AI the World's Leading AI and Technology News and Media Company. If you are building an AI-related product or service, we invite you to consider becoming an AI sponsor. At Towards AI, we help scale AI and technology startups. Let us help you unleash your technology to the masses. "The environment selects those few mutations that enhance survival, resulting in a series of slow transformations of one lifeform into another, the origin of a new species."-

For example, in [16], the authors discovered become automated and more "end-to-end," the data augmentation that SpecAugment [11] did not compose well with multi-style policy (what augmentation functions to use, and how training augmentation [4, 17], and found that they needed to to apply them) remains hand-crafted. We present G(raph)- ensemble the augmentations to benefit from both. Augment, a technique to define the augmentation space as In this work, we address this problem by a scheme we refer directed acyclic graphs (DAGs) and search over this space to as G(raph)-Augment, where a stochastic augmentation to optimize the augmentation policy itself. We show that policy is parameterized by a directed acyclic graph (DAG) given the same computational budget, policies produced by whose edges are labeled by sampling probabilities and augmentation G-Augment are able to perform better than SpecAugment parameters. By simultaneously searching for the policies obtained by random search on fine-tuning tasks on graph structure and the parameters that label the graph, we CHiME-6 and AMI. G-Augment is also able to establish are able to optimize not only the augmentation parameters of a new state-of-the-art ASR performance on the CHiME-6 the individual augmentations, but how those augmentations evaluation set (30.7% WER). We further demonstrate that are being applied. We utilize 17 ASR augmentations in our G-Augment policies show better transfer properties across search space, details of which can be found in section 3.3.

Filter pruning is a common method to achieve model compression and acceleration in deep neural networks (DNNs).Some research regarded filter pruning as a combinatorial optimization problem and thus used evolutionary algorithms (EA) to prune filters of DNNs. However, it is difficult to find a satisfactory compromise solution in a reasonable time due to the complexity of solution space searching. To solve this problem, we first formulate a multi-objective optimization problem based on a sub-network of the full model and propose a Sub-network Multiobjective Evolutionary Algorithm (SMOEA) for filter pruning. By progressively pruning the convolutional layers in groups, SMOEA can obtain a lightweight pruned result with better performance.Experiments on VGG-14 model for CIFAR-10 verify the effectiveness of the proposed SMOEA. Specifically, the accuracy of the pruned model with 16.56% parameters decreases by 0.28% only, which is better than the widely used popular filter pruning criteria.

A high-dimensional and incomplete (HDI) matrix frequently appears in various big-data-related applications, which demonstrates the inherently non-negative interactions among numerous nodes. A non-negative latent factor (NLF) model performs efficient representation learning to an HDI matrix, whose learning process mostly relies on a single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) algorithm. However, an SLF-NMU algorithm updates a latent factor based on the current update increment only without appropriate considerations of past learning information, resulting in slow convergence. Inspired by the prominent success of a proportional-integral (PI) controller in various applications, this paper proposes a Proportional-Integral-incorporated Non-negative Latent Factor (PI-NLF) model with two-fold ideas: a) establishing an Increment Refinement (IR) mechanism via considering the past update increments following the principle of a PI controller; and b) designing an IR-based SLF-NMU (ISN) algorithm to accelerate the convergence rate of a resultant model. Empirical studies on four HDI datasets demonstrate that a PI-NLF model outperforms the state-of-the-art models in both computational efficiency and estimation accuracy for missing data of an HDI matrix. Hence, this study unveils the feasibility of boosting the performance of a non-negative learning algorithm through an error feedback controller.

High-Dimensional and Incomplete (HDI) data are frequently found in various industrial applications with complex interactions among numerous nodes, which are commonly non-negative for representing the inherent non-negativity of node interactions. A Non-negative Latent Factor (NLF) model is able to extract intrinsic features from such data efficiently. However, existing NLF models all adopt a static divergence metric like Euclidean distance or {\alpha}-\b{eta} divergence to build its learning objective, which greatly restricts its scalability of accurately representing HDI data from different domains. Aiming at addressing this issue, this study presents an Adaptive Divergence-based Non-negative Latent Factor (ADNLF) model with three-fold ideas: a) generalizing the objective function with the {\alpha}-\b{eta}-divergence to expand its potential of representing various HDI data; b) adopting a non-negative bridging function to connect the optimization variables with output latent factors for fulfilling the non-negativity constraints constantly; and c) making the divergence parameters adaptive through particle swarm optimization, thereby facilitating adaptive divergence in the learning objective to achieve high scalability. Empirical studies are conducted on four HDI datasets from real applications, whose results demonstrate that in comparison with state-of-the-art NLF models, an ADNLF model achieves significantly higher estimation accuracy for missing data of an HDI dataset with high computational efficiency.

Derivative-free prompt learning has emerged as a lightweight alternative to prompt tuning, which only requires model inference to optimize the prompts. However, existing work did not take full advantage of the over-parameterized characteristics of large pre-trained language models (PLMs). In this paper, we propose Clip-Tuning, a simple yet effective method that adopts diverse frozen "thinned" networks of PLMs to obtain a mixture of rewards and thus advance the derivative-free prompt learning. The thinned networks consist of all the hidden units that survive a stationary dropout strategy, whose inference predictions reflect an ensemble of partial views over prompted training samples. Our method outperforms previous gradient-free prompt learning methods and achieves parity with gradient-based counterparts on seven language understanding benchmarks under few-shot settings.

Feature selection plays a vital role in promoting the classifier's performance. However, current methods ineffectively distinguish the complex interaction in the selected features. To further remove these hidden negative interactions, we propose a GA-like dynamic probability (GADP) method with mutual information which has a two-layer structure. The first layer applies the mutual information method to obtain a primary feature subset. The GA-like dynamic probability algorithm, as the second layer, mines more supportive features based on the former candidate features. Essentially, the GA-like method is one of the population-based algorithms so its work mechanism is similar to the GA. Different from the popular works which frequently focus on improving GA's operators for enhancing the search ability and lowering the converge time, we boldly abandon GA's operators and employ the dynamic probability that relies on the performance of each chromosome to determine feature selection in the new generation. The dynamic probability mechanism significantly reduces the parameter number in GA that making it easy to use. As each gene's probability is independent, the chromosome variety in GADP is more notable than in traditional GA, which ensures GADP has a wider search space and selects relevant features more effectively and accurately. To verify our method's superiority, we evaluate our method under multiple conditions on 15 datasets. The results demonstrate the outperformance of the proposed method. Generally, it has the best accuracy. Further, we also compare the proposed model to the popular heuristic methods like POS, FPA, and WOA. Our model still owns advantages over them.

Physics-informed neural networks (PINN) have recently emerged as a promising application of deep learning in a wide range of engineering and scientific problems based on partial differential equation (PDE) models. However, evidence shows that PINN training by gradient descent displays pathologies that often prevent convergence when solving PDEs with irregular solutions. In this paper, we propose the use of a particle swarm optimization (PSO) approach to train PINNs. The resulting PSO-PINN algorithm not only mitigates the undesired behaviors of PINNs trained with standard gradient descent but also presents an ensemble approach to PINN that affords the possibility of robust predictions with quantified uncertainty. We also propose PSO-BP-CD (PSO with Back-Propagation and Coefficient Decay), a hybrid PSO variant that combines swarm optimization with gradient descent, putting more weight on the latter as training progresses and the swarm zeros in on a good local optimum. Comprehensive experimental results show that PSO-PINN with the proposed PSO-BP-CD algorithm outperforms PINN ensembles trained with other PSO variants or with pure gradient descent.

This paper reviews various Evolutionary Approaches applied to the domain of Evolutionary Robotics with the intention of resolving difficult problems in the areas of robotic design and control. Evolutionary Robotics is a fast-growing field that has attracted substantial research attention in recent years. The paper thus collates recent findings along with some anticipated applications. The reviewed literature is organized systematically to give a categorical overview of recent developments and is presented in tabulated form for quick reference. We discuss the outstanding potentialities and challenges that exist in robotics from an ER perspective, with the belief that these will be have the capacity to be addressed in the near future via the application of evolutionary approaches. The primary objective of this study is to explore the applicability of Evolutionary Approaches in robotic application development. We believe that this study will enable the researchers to utilize Evolutionary Approaches to solve complex outstanding problems in robotics.