The Multi-Prize Lottery Ticket Hypothesis posits that randomly initialized neural networks contain several subnetworks that achieve comparable accuracy to fully trained models of the same architecture. However, current methods require that the network is sufficiently overparameterized. In this work, we propose a modification to two state-of-the-art algorithms (Edge-Popup and Biprop) that finds high-accuracy subnetworks with no additional storage cost or scaling. The algorithm, Iterative Weight Recycling, identifies subsets of important weights within a randomly initialized network for intra-layer reuse. Empirically we show improvements on smaller network architectures and higher prune rates, finding that model sparsity can be increased through the "recycling" of existing weights. In addition to Iterative Weight Recycling, we complement the Multi-Prize Lottery Ticket Hypothesis with a reciprocal finding: high-accuracy, randomly initialized subnetwork's produce diverse masks, despite being generated with the same hyperparameter's and pruning strategy. We explore the landscapes of these masks, which show high variability.

Detecting malicious activity within an enterprise computer network can be framed as a temporal link prediction task: given a sequence of graphs representing communications between hosts over time, the goal is to predict which edges should--or should not--occur in the future. However, standard temporal link prediction algorithms are ill-suited for computer network monitoring as they do not take account of the peculiar short-term dynamics of computer network activity, which exhibits sharp seasonal variations. In order to build a better model, we propose a source separation-inspired description of computer network activity: at each time step, the observed graph is a mixture of subgraphs representing various sources of activity, and short-term dynamics result from changes in the mixing coefficients. Both qualitative and quantitative experiments demonstrate the validity of our approach.

My team's theoretical and clinical research approach is based on the use of descriptive and predictive mathematical modeling. This has recently been complemented with AI-related resources such as supervised and unsupervised learning. Our goal is twofold: (1) to better understand data collected using biometric and imaging techniques and (2) to increase the diagnostic accuracy or effectiveness of corrective and refractive therapeutic solutions. Thus far, the models and techniques have been applied mainly to refractive surgery, topography and corneal imaging for keratoconus screening, ocular wavefront description, complex optical comparison and design in cataract surgery, ocular biometry, and IOL calculation. The improvements to diagnostic tools include screening or characterization of keratoconus, corneal edema, and ocular wavefront; the improvements to therapeutic tools include ablation profiles, IOL calculations, and diffractive optics.

Silicon Valley churns out new products all the time, but rarely does one receive the level of hype that has surrounded the release of GPT-4. The follow-up to ChatGPT can ace standardized tests, tell you why a meme is funny, and even help do your taxes. Since the San Francisco start-up OpenAI introduced the technology earlier this month, it has been branded as "remarkable but unsettling," and has led to grandiose statements about how "things will never be the same." But actually trying out these features for yourself--or at least the ones that have already been publicly released--does not come cheap. Unlike ChatGPT, which captivated the world because it was free, GPT-4 is currently only available to non-developers through a premium service that costs $20 a month.

ChatGPT is one of the most advanced and popular chatbots in the world, powered by OpenAI's generative pre-trained transformer (GPT) models. ChatGPT can generate natural and engaging responses to a variety of topics and contexts, using its large-scale language understanding and generation capabilities. However, until recently, ChatGPT had a major limitation: it could not access the internet or any external data sources or services. This meant that ChatGPT could only rely on its internal knowledge and memory, which might be outdated, incomplete, or inaccurate. For example, if you asked ChatGPT about the latest news, sports scores, or weather forecasts, it would not be able to give you a reliable answer.

There's a common perception that artificial intelligence (AI) will help streamline our work. There are even fears that it could wipe out the need for some jobs altogether. But in a study of science laboratories I carried out with three colleagues at the University of Manchester, the introduction of automated processes that aim to simplify work--and free people's time--can also make that work more complex, generating new tasks that many workers might perceive as mundane. In the study, published in Research Policy, we looked at the work of scientists in a field called synthetic biology, or synbio for short. Synbio is concerned with redesigning organisms to have new abilities.

Deep neural architectures have profound impact on achieved performance in many of today's AI tasks, yet, their design still heavily relies on human prior knowledge and experience. Neural architecture search (NAS) together with hyperparameter optimization (HO) helps to reduce this dependence. However, state of the art NAS and HO rapidly become infeasible with increasing amount of data being stored in a distributed fashion, typically violating data privacy regulations such as GDPR and CCPA. As a remedy, we introduce FEATHERS - $\textbf{FE}$derated $\textbf{A}$rchi$\textbf{T}$ecture and $\textbf{H}$yp$\textbf{ER}$parameter $\textbf{S}$earch, a method that not only optimizes both neural architectures and optimization-related hyperparameters jointly in distributed data settings, but further adheres to data privacy through the use of differential privacy (DP). We show that FEATHERS efficiently optimizes architectural and optimization-related hyperparameters alike, while demonstrating convergence on classification tasks at no detriment to model performance when complying with privacy constraints.

Word alignments are essential for a variety of NLP tasks. Therefore, choosing the best approaches for their creation is crucial. However, the scarce availability of gold evaluation data makes the choice difficult. We propose SilverAlign, a new method to automatically create silver data for the evaluation of word aligners by exploiting machine translation and minimal pairs. We show that performance on our silver data correlates well with gold benchmarks for 9 language pairs, making our approach a valid resource for evaluation of different domains and languages when gold data are not available. This addresses the important scenario of missing gold data alignments for low-resource languages.

Making sense of familiar yet new situations typically involves making generalizations about causal schemas, stories that help humans reason about event sequences. Reasoning about events includes identifying cause and effect relations shared across event instances, a process we refer to as causal schema induction. Statistical schema induction systems may leverage structural knowledge encoded in discourse or the causal graphs associated with event meaning, however resources to study such causal structure are few in number and limited in size. In this work, we investigate how to apply schema induction models to the task of knowledge discovery for enhanced search of English-language news texts. To tackle the problem of data scarcity, we present Torquestra, a manually curated dataset of text-graph-schema units integrating temporal, event, and causal structures. We benchmark our dataset on three knowledge discovery tasks, building and evaluating models for each. Results show that systems that harness causal structure are effective at identifying texts sharing similar causal meaning components rather than relying on lexical cues alone. We make our dataset and models available for research purposes.

Abstract--Machine learning methods strive to acquire a robust model during training that can generalize well to test samples, even under distribution shifts. However, these methods often suffer from a performance drop due to unknown test distributions. Test-time adaptation (TTA), an emerging paradigm, has the potential to adapt a pre-trained model to unlabeled data during testing, before making predictions. Recent progress in this paradigm highlights the significant benefits of utilizing unlabeled data for training self-adapted models prior to inference. In this survey, we divide TTA into several distinct categories, namely, test-time (source-free) domain adaptation, test-time batch adaptation, online test-time adaptation, and test-time prior adaptation. For each category, we provide a comprehensive taxonomy of advanced algorithms, followed by a discussion of different learning scenarios. Furthermore, we analyze relevant applications of TTA and discuss open challenges and promising areas for future research. However, when the test distribution (target) differs from the training distribution (source), we face the problem of distribution shifts. Such a shift poses significant challenges for machine learning systems deployed in the wild, such as images captured by different cameras [2], road scenes of different cities [3], and imaging devices in different hospitals [4]. In contrast, TTA only requires access to the pre-trained from one or multiple source domains that can generalize model from the source domain, making it a secure and well to any out-of-distribution target domain. Figure 1: test-time domain adaptation, test-time batch adaptation This survey primarily focuses on test-time adaptation (TTBA), and online test-time adaptation (OTTA). That is to say, test data. Additionally, DA typically necessitates access to the predictions of each mini-batch are independent of the both labeled data from the source domain and (unlabeled) predictions for the other mini-batches. Ran He is also with the School of Artificial Intelligence, University of Chinese Academy of Sciences. In this survey, we use the terms "test data" and "target data" Tieniu Tan is also with Nanjing University, China. DA methods rely on the existence of source applied to OTTA with the assumption of knowledge reuse.