Hierarchical learning algorithms that gradually approximate a solution to a data-driven optimization problem are essential to decision-making systems, especially under limitations on time and computational resources. In this study, we introduce a general-purpose hierarchical learning architecture that is based on the progressive partitioning of a possibly multi-resolution data space. The optimal partition is gradually approximated by solving a sequence of optimization sub-problems that yield a sequence of partitions with increasing number of subsets. We show that the solution of each optimization problem can be estimated online using gradient-free stochastic approximation updates. As a consequence, a function approximation problem can be defined within each subset of the partition and solved using the theory of two-timescale stochastic approximation algorithms. This simulates an annealing process and defines a robust and interpretable heuristic method to gradually increase the complexity of the learning architecture in a task-agnostic manner, giving emphasis to regions of the data space that are considered more important according to a predefined criterion. Finally, by imposing a tree structure in the progression of the partitions, we provide a means to incorporate potential multi-resolution structure of the data space into this approach, significantly reducing its complexity, while introducing hierarchical variable-rate feature extraction properties similar to certain classes of deep learning architectures. Asymptotic convergence analysis and experimental results are provided for supervised and unsupervised learning problems.

Designing neural architectures requires immense manual efforts. This has promoted the development of neural architecture search (NAS) to automate the design. While previous NAS methods achieve promising results but run slowly, zero-cost proxies run extremely fast but are less promising. Therefore, it is of great potential to accelerate NAS via those zero-cost proxies. The existing method has two limitations, which are unforeseeable reliability and one-shot usage. To address the limitations, we present ProxyBO, an efficient Bayesian optimization (BO) framework that utilizes the zero-cost proxies to accelerate neural architecture search. We apply the generalization ability measurement to estimate the fitness of proxies on the task during each iteration and design a novel acquisition function to combine BO with zero-cost proxies based on their dynamic influence. Extensive empirical studies show that ProxyBO consistently outperforms competitive baselines on five tasks from three public benchmarks. Concretely, ProxyBO achieves up to 5.41x and 3.86x speedups over the state-of-the-art approaches REA and BRP-NAS.

Dextrous in-hand manipulation with a multi-fingered robotic hand is a challenging task, esp. when performed with the hand oriented upside down, demanding permanent force-closure, and when no external sensors are used. For the task of reorienting an object to a given goal orientation (vs. infinitely spinning it around an axis), the lack of external sensors is an additional fundamental challenge as the state of the object has to be estimated all the time, e.g., to detect when the goal is reached. In this paper, we show that the task of reorienting a cube to any of the 24 possible goal orientations in a ${\pi}$/2-raster using the torque-controlled DLR-Hand II is possible. The task is learned in simulation using a modular deep reinforcement learning architecture: the actual policy has only a small observation time window of 0.5s but gets the cube state as an explicit input which is estimated via a deep differentiable particle filter trained on data generated by running the policy. In simulation, we reach a success rate of 92% while applying significant domain randomization. Via zero-shot Sim2Real-transfer on the real robotic system, all 24 goal orientations can be reached with a high success rate.

The development of smart city transport systems, including self-driving cars, leads to an increase in the threat of hostile interference in the processes of vehicle control. This interference may disrupt the normal functioning of the transport system, and, if is performed covertly, the system can be negatively affected for a long period of time. This paper develops a simulation stochastic cellular automata model of traffic on a circular two-lane road based on the Sakai-Nishinari-Fukui-Schadschneider (S-NFS) rules. In the presented model, in addition to ordinary vehicles, there are covertly counteracting vehicles; their task is to reduce the quantity indicators (such as traffic flux) of the transport system using special rules of behavior. Three such rules are considered and compared: two lane-changing rules and one slow-down rule. It is shown that such counteracting vehicles can affect the traffic flow, mainly in the region of the maximum of the fundamental diagram, that is, at average values of the vehicle density. In free-flowing traffic or in a traffic jam, the influence of the counteracting vehicle is negligible regardless of its rules of behavior.

Cellular automata (CAs) are computational models exhibiting rich dynamics emerging from the local interaction of cells arranged in a regular lattice. Graph CAs (GCAs) generalise standard CAs by allowing for arbitrary graphs rather than regular lattices, similar to how Graph Neural Networks (GNNs) generalise Convolutional NNs. Recently, Graph Neural CAs (GNCAs) have been proposed as models built on top of standard GNNs that can be trained to approximate the transition rule of any arbitrary GCA. Existing GNCAs are anisotropic in the sense that their transition rules are not equivariant to translation, rotation, and reflection of the nodes' spatial locations. However, it is desirable for instances related by such transformations to be treated identically by the model. By replacing standard graph convolutions with E(n)-equivariant ones, we avoid anisotropy by design and propose a class of isotropic automata that we call E(n)-GNCAs. These models are lightweight, but can nevertheless handle large graphs, capture complex dynamics and exhibit emergent self-organising behaviours. We showcase the broad and successful applicability of E(n)-GNCAs on three different tasks: (i) pattern formation, (ii) graph auto-encoding, and (iii) simulation of E(n)-equivariant dynamical systems.

In this paper, we present a modified Xception architecture, the NEXcepTion network. Our network has significantly better performance than the original Xception, achieving top-1 accuracy of 81.5% on the ImageNet validation dataset (an improvement of 2.5%) as well as a 28% higher throughput. Another variant of our model, NEXcepTion-TP, reaches 81.8% top-1 accuracy, similar to ConvNeXt (82.1%), while having a 27% higher throughput. Our model is the result of applying improved training procedures and new design decisions combined with an application of Neural Architecture Search (NAS) on a smaller dataset. These findings call for revisiting older architectures and reassessing their potential when combined with the latest enhancements.

Significantly higher transmissibility of SARS CoV-2 (2019) compared to SARS CoV (2003) can be attributed to mutations of structural proteins (Spike S, Nucleocapsid N, Membrane M, and Envelope E) and the role played by non-structural proteins (nsps) and accessory proteins (ORFs) for viral replication, assembly and shedding. The non-structural proteins (nsps) avail host protein synthesis machinery to initiate viral replication, along with neutralization of host immune defense. The key protein out of the 16 nsps, is the non-structural protein nsp1, also known as the leader protein. Nsp1 leads the process of hijacking host resources by blocking host translation. This paper concentrates on the analysis of nsps of SARS covid (CoV-2, CoV) and MERS covid based on Cellular Automata enhanced Machine Learning (CAML) model developed for study of biological strings. This computational model compares deviation of structure - function of CoV-2 from that of CoV employing CAML model parameters derived out of CA evolution of amino acid chains of nsps. This comparative analysis points to - (i) higher transmissibility of CoV-2 compared to CoV for major nsps, and (ii) deviation of MERS covid from SARS CoV in respect of virulence and pathogenesis. A Machine Learning (ML) framework has been designed to map the CAML model parameters to the physical domain features reported in in-vitro/in-vivo/in-silico experimental studies. The ML framework enables us to learn the permissible range of model parameters derived out of mutational study of sixteen nsps of three viruses.

Federated Learning (FL) is extensively used to train AI/ML models in distributed and privacy-preserving settings. Participant edge devices in FL systems typically contain non-independent and identically distributed~(Non-IID) private data and unevenly distributed computational resources. Preserving user data privacy while optimizing AI/ML models in a heterogeneous federated network requires us to address data heterogeneity and system/resource heterogeneity. Hence, we propose \underline{R}esource-\underline{a}ware \underline{F}ederated \underline{L}earning~(RaFL) to address these challenges. RaFL allocates resource-aware models to edge devices using Neural Architecture Search~(NAS) and allows heterogeneous model architecture deployment by knowledge extraction and fusion. Integrating NAS into FL enables on-demand customized model deployment for resource-diverse edge devices. Furthermore, we propose a multi-model architecture fusion scheme allowing the aggregation of the distributed learning results. Results demonstrate RaFL's superior resource efficiency compared to SoTA.

Recent extensions of Cellular Automata (CA) have incorporated key ideas from modern deep learning, dramatically extending their capabilities and catalyzing a new family of Neural Cellular Automata (NCA) techniques. Inspired by Transformer-based architectures, our work presents a new class of $\textit{attention-based}$ NCAs formed using a spatially localized$\unicode{x2014}$yet globally organized$\unicode{x2014}$self-attention scheme. We introduce an instance of this class named $\textit{Vision Transformer Cellular Automata}$ (ViTCA). We present quantitative and qualitative results on denoising autoencoding across six benchmark datasets, comparing ViTCA to a U-Net, a U-Net-based CA baseline (UNetCA), and a Vision Transformer (ViT). When comparing across architectures configured to similar parameter complexity, ViTCA architectures yield superior performance across all benchmarks and for nearly every evaluation metric. We present an ablation study on various architectural configurations of ViTCA, an analysis of its effect on cell states, and an investigation on its inductive biases. Finally, we examine its learned representations via linear probes on its converged cell state hidden representations, yielding, on average, superior results when compared to our U-Net, ViT, and UNetCA baselines.

The high-dimesionality, non-linearity and emergent properties of complex systems pose a challenge to identifying general laws in the same manner that has been so successful in simpler physical systems. In Anderson's seminal work on why "more is different" he pointed to how emergent, macroscale patterns break symmetries of the underlying microscale laws. Yet, less recognized is that these large-scale, emergent patterns must also retain some symmetries of the microscale rules. Here we introduce a new, relational macrostate theory (RMT) that defines macrostates in terms of symmetries between two mutually predictive observations, and develop a machine learning architecture, MacroNet, that identifies macrostates. Using this framework, we show how macrostates can be identifed across systems ranging in complexity from the simplicity of the simple harmonic oscillator to the much more complex spatial patterning characteristic of Turing instabilities. Furthermore, we show how our framework can be used for the inverse design of microstates consistent with a given macroscopic property -- in Turing patterns this allows us to design underlying rule with a given specification of spatial patterning, and to identify which rule parameters most control these patterns. By demonstrating a general theory for how macroscopic properties emerge from conservation of symmetries in the mapping between observations, we provide a machine learning framework that allows a unified approach to identifying macrostates in systems from the simple to complex, and allows the design of new examples consistent with a given macroscopic property.