We propose an automaton model which is a combination of symbolic and register automata, i.e., we enrich symbolic automata with memory. We call such automata Symbolic Register Automata (SRA). SRA extend the expressive power of symbolic automata, by allowing Boolean formulas to be applied not only to the last element read from the input string, but to multiple elements, stored in their registers. SRA also extend register automata, by allowing arbitrary Boolean formulas, besides equality predicates. We study the closure properties of SRA under union, intersection, concatenation, Kleene closure, complement and determinization and show that SRA, contrary to symbolic automata, are not in general closed under complement and they are not determinizable. However, they are closed under these operations when a window operator, quintessential in Complex Event Recognition, is used. We show how SRA can be used in Complex Event Recognition in order to detect patterns upon streams of events, using our framework that provides declarative and compositional semantics, and that allows for a systematic treatment of such automata. We also show how the behavior of SRA, as they consume streams of events, can be given a probabilistic description with the help of prediction suffix trees. This allows us to go one step beyond Complex Event Recognition to Complex Event Forecasting, where, besides detecting complex patterns, we can also efficiently forecast their occurrence.

In cooperative bandits, a framework that captures essential features of collective sequential decision making, agents can minimize group regret, and thereby improve performance, by leveraging shared information. However, sharing information can be costly, which motivates developing policies that minimize group regret while also reducing the number of messages communicated by agents. Existing cooperative bandit algorithms obtain optimal performance when agents share information with their neighbors at \textit{every time step}, i.e., full communication. This requires $\Theta(T)$ number of messages, where $T$ is the time horizon of the decision making process. We propose \textit{ComEx}, a novel cost-effective communication protocol in which the group achieves the same order of performance as full communication while communicating only $O(\log T)$ number of messages. Our key step is developing a method to identify and only communicate the information crucial to achieving optimal performance. Further we propose novel algorithms for several benchmark cooperative bandit frameworks and show that our algorithms obtain \textit{state-of-the-art} performance while consistently incurring a significantly smaller communication cost than existing algorithms.

Patent offices and courts around the world are being asked to tackle a similar question: can an artificial intelligence system qualify as an inventor for a patent? A test case making its way through several countries--from Saudi Arabia to Australia to Brazil--has spurred debate about advancements in artificial intelligence technology and questions about whether patent laws need to be revised to recognize machines as inventors. A judge in the U.S. District Court for the Eastern District of Virginia recently ruled that, under current U.S. law, AI can't be listed as an inventor on a patent. The ruling was in line with what U.S., British, and EU patent officials have concluded. The push to recognize AI as an inventor comes from Ryan Abbott, a University of Surrey law professor, and Stephen Thaler, a computer scientist from Missouri.

Ukraine said it will build a factory to produce Turkish armed drones that Kyiv previously bought to use against pro-Russian separatists in the east, a deal that might upset Kyiv's adversary Moscow. "A land plot on which the factory will be built has already been chosen," Ukrainian Foreign Minister Dmytro Kuleba said at a news conference on Thursday with Turkish counterpart Mevlut Cavusoglu in the western Ukrainian city of Lviv. "There were a number of obstacles to the implementation [of this project] but all of them have been removed," he added, without providing further details. Pleased to welcome my Turkish colleague and friend @MevlutCavusoglu in Lviv and expand our diplomatic geography. Cavusoglu did not speak specifically about the subject but stressed that Kyiv and Ankara were "in the process of strengthening their relations in many sectors", including defence.

Pangolins, also known as scaly anteaters, are currently the most trafficked mammal species. Criminals can be resourceful and unrelenting in their efforts to find a way around obstacles. Wildlife traffickers are no exception. Today's trade in wildlife and wildlife products has shifted from physical markets to online marketplaces where traffickers apply e-commerce business models and use encrypted messages in an attempt to evade detection by law enforcement. While the move towards online platforms started several years before the Covid-19 pandemic, the restrictions imposed to contain the virus accelerated this digital transformation.

Earlier this month, a federal district court issued the first judicial decision in the country addressing whether an AI system can be an "inventor" under U.S. patent law. The decision was rendered by the U.S. District Court for the Eastern District of Virginia in Thaler v. Hirshfeld on appeal from the U.S. Patent and Trademark Office's (USPTO) decision that refused to allow Thaler's two patent applications to proceed because he listed DABUS (an AI machine) as the inventor. Thaler filed the applications in 2018--one for an invention used to contain food and the other for a flashing beacon for attracting attention in emergencies. In statements filed in support of the applications, Thaler listed DABUS as the inventor, claiming that he had acquired the right to the grant of the patents by "ownership of the creativity machine." In affirming the USPTO's denial of the applications, the court held that based on the plain statutory language of the U.S. Patent Act and Federal Circuit authority, an AI machine cannot be an inventor because an inventor must be an "individual," which under common interpretation and court precedent means a natural person. The court stated that Thaler's argument was based on policy considerations and the purpose of the patent clause of the U.S. Constitution, and that the decision to expand the scope of inventorship is squarely within the authority of Congress.

Pruning is a model compression method that removes redundant parameters in deep neural networks (DNNs) while maintaining accuracy. Most available filter pruning methods require complex treatments such as iterative pruning, features statistics/ranking, or additional optimization designs in the training process. In this paper, we propose a simple and effective regularization strategy from a new perspective of evolution of features, which we call feature flow regularization (FFR), for improving structured sparsity and filter pruning in DNNs. Specifically, FFR imposes controls on the gradient and curvature of feature flow along the neural network, which implicitly increases the sparsity of the parameters. The principle behind FFR is that coherent and smooth evolution of features will lead to an efficient network that avoids redundant parameters. The high structured sparsity obtained from FFR enables us to prune filters effectively. Experiments with VGGNets, ResNets on CIFAR-10/100, and Tiny ImageNet datasets demonstrate that FFR can significantly improve both unstructured and structured sparsity. Our pruning results in terms of reduction of parameters and FLOPs are comparable to or even better than those of state-of-the-art pruning methods.

We introduce the algorithm Bayesian Optimization (BO) with Fictitious Play (BOFiP) for the optimization of high dimensional black box functions. BOFiP decomposes the original, high dimensional, space into several sub-spaces defined by non-overlapping sets of dimensions. These sets are randomly generated at the start of the algorithm, and they form a partition of the dimensions of the original space. BOFiP searches the original space with alternating BO, within sub-spaces, and information exchange among sub-spaces, to update the sub-space function evaluation. The basic idea is to distribute the high dimensional optimization across low dimensional sub-spaces, where each sub-space is a player in an equal interest game. At each iteration, BO produces approximate best replies that update the players belief distribution. The belief update and BO alternate until a stopping condition is met. High dimensional problems are common in real applications, and several contributions in the BO literature have highlighted the difficulty in scaling to high dimensions due to the computational complexity associated to the estimation of the model hyperparameters. Such complexity is exponential in the problem dimension, resulting in substantial loss of performance for most techniques with the increase of the input dimensionality. We compare BOFiP to several state-of-the-art approaches in the field of high dimensional black box optimization. The numerical experiments show the performance over three benchmark objective functions from 20 up to 1000 dimensions. A neural network architecture design problem is tested with 42 up to 911 nodes in 6 up to 92 layers, respectively, resulting into networks with 500 up to 10,000 weights. These sets of experiments empirically show that BOFiP outperforms its competitors, showing consistent performance across different problems and increasing problem dimensionality.

Federated learning (FL) is a privacy-preserving paradigm where multiple participants jointly solve a machine learning problem without sharing raw data. Unlike traditional distributed learning, a unique characteristic of FL is statistical heterogeneity, namely, data distributions across participants are different from each other. Meanwhile, recent advances in the interpretation of neural networks have seen a wide use of neural tangent kernel (NTK) for convergence and generalization analyses. In this paper, we propose a novel FL paradigm empowered by the NTK framework. The proposed paradigm addresses the challenge of statistical heterogeneity by transmitting update data that are more expressive than those of the traditional FL paradigms. Specifically, sample-wise Jacobian matrices, rather than model weights/gradients, are uploaded by participants. The server then constructs an empirical kernel matrix to update a global model without explicitly performing gradient descent. We further develop a variant with improved communication efficiency and enhanced privacy. Numerical results show that the proposed paradigm can achieve the same accuracy while reducing the number of communication rounds by an order of magnitude compared to federated averaging.

In this paper, we provide the first focused study on the discontinuities (aka. When investigating latent holes, existing works are exclusively centred around the encoder network and they merely explore the existence of holes. We tackle these limitations by proposing a highly efficient Tree-based Decoder-Centric (TDC) algorithm for latent hole identification, with a focal point on the text domain. In contrast to past studies, our approach pays attention to the decoder network, as a decoder has a direct impact on the model's output quality. Furthermore, we provide, for the first time, in-depth empirical analysis of the latent hole phenomenon, investigating several important aspects such as how the holes impact VAE algorithms' performance on text generation, and how the holes are distributed in the latent space.