Continual learning is a sub-field of machine learning, which aims to allow machine learning models to continuously learn on new data, by accumulating knowledge without forgetting what was learned in the past. In this work, we take a step back, and ask: "Why should one care about continual learning in the first place?". We set the stage by surveying recent continual learning papers published at three major machine learning conferences, and show that memory-constrained settings dominate the field. Then, we discuss five open problems in machine learning, and even though they seem unrelated to continual learning at first sight, we show that continual learning will inevitably be part of their solution. These problems are model-editing, personalization, on-device learning, faster (re-)training and reinforcement learning. Finally, by comparing the desiderata from these unsolved problems and the current assumptions in continual learning, we highlight and discuss four future directions for continual learning research. We hope that this work offers an interesting perspective on the future of continual learning, while displaying its potential value and the paths we have to pursue in order to make it successful. This work is the result of the many discussions the authors had at the Dagstuhl seminar on Deep Continual Learning, in March 2023.

Deep learning models often suffer from forgetting previously learned information when trained on new data. This problem is exacerbated in federated learning (FL), where the data is distributed and can change independently for each user. Many solutions are proposed to resolve this catastrophic forgetting in a centralized setting. However, they do not apply directly to FL because of its unique complexities, such as privacy concerns and resource limitations. To overcome these challenges, this paper presents a framework for $\textbf{federated class incremental learning}$ that utilizes a generative model to synthesize samples from past distributions. This data can be later exploited alongside the training data to mitigate catastrophic forgetting. To preserve privacy, the generative model is trained on the server using data-free methods at the end of each task without requesting data from clients. Moreover, our solution does not demand the users to store old data or models, which gives them the freedom to join/leave the training at any time. Additionally, we introduce SuperImageNet, a new regrouping of the ImageNet dataset specifically tailored for federated continual learning. We demonstrate significant improvements compared to existing baselines through extensive experiments on multiple datasets.

We develop a pairing-based graph neural network for simulating quantum many-body systems. Our architecture augments a BCS-type geminal wavefunction with a generalized pair amplitude parameterized by a graph neural network. Variational Monte Carlo with our neural network simultaneously provides an accurate, flexible, and scalable method for simulating many-electron systems. We apply this method to two-dimensional semiconductor electron-hole bilayers and obtain accurate results on a variety of interaction-induced phases, including the exciton Bose-Einstein condensate, electron-hole superconductor, and bilayer Wigner crystal. Our study demonstrates the potential of physically-motivated neural network wavefunctions for quantum materials simulations.

As the cost associated with fine-tuning Large Language Models (LLMs) continues to rise, recent research efforts have pivoted towards developing methodologies to edit implicit knowledge embedded within LLMs. Yet, there's still a dark cloud lingering overhead -- will knowledge editing trigger butterfly effect? since it is still unclear whether knowledge editing might introduce side effects that pose potential risks or not. This paper pioneers the investigation into the potential pitfalls associated with knowledge editing for LLMs. To achieve this, we introduce new benchmark datasets and propose innovative evaluation metrics. Our results underline two pivotal concerns: (1) Knowledge Conflict: Editing groups of facts that logically clash can magnify the inherent inconsistencies in LLMs-a facet neglected by previous methods. (2) Knowledge Distortion: Altering parameters with the aim of editing factual knowledge can irrevocably warp the innate knowledge structure of LLMs. Experimental results vividly demonstrate that knowledge editing might inadvertently cast a shadow of unintended consequences on LLMs, which warrant attention and efforts for future works. Code is available at https://github.com/zjunlp/PitfallsKnowledgeEditing.

In general, low-fidelity data is easier to obtain in greater quantities, but it may be too inaccurate or not dense enough to accurately train a machine learning model. High-fidelity data is costly to obtain, so there may not be sufficient data to use in training, however, it is more accurate. A small amount of high fidelity data, such as from measurements, combined with low fidelity data, can improve predictions when used together; this has motivated geophysicists to develop cokriging [1], which is based on Gaussian process regression at two different fidelity levels by exploiting correlations-albeit only linear ones - between different levels. An example of cokriging for obtaining the sea surface temperature (as well as the associated uncertainty) is presented in [2], where satellite images are used as low-fidelity data whereas in situ measurements are used as high-fidelity data. To exploit nonlinear correlations at different levels of fidelity, a probabilistic framework based on Gaussian process regression and nonlinear autoregressive scheme was proposed in [3] that can learn complex nonlinear and space-dependent cross-correlations between multifidelity models. However, the limitation of this work is the high computational cost for big data sets, and to this end, the subsequent work in [4] was based on neural networks and provided the first method of multifidelity training of deep neural networks.

Bayesian predictive synthesis (BPS) provides a method for combining multiple predictive distributions based on agent/expert opinion analysis theory and encompasses a range of existing density forecast pooling methods. The key ingredient in BPS is a ``synthesis'' function. This is typically specified parametrically as a dynamic linear regression. In this paper, we develop a nonparametric treatment of the synthesis function using regression trees. We show the advantages of our tree-based approach in two macroeconomic forecasting applications. The first uses density forecasts for GDP growth from the euro area's Survey of Professional Forecasters. The second combines density forecasts of US inflation produced by many regression models involving different predictors. Both applications demonstrate the benefits -- in terms of improved forecast accuracy and interpretability -- of modeling the synthesis function nonparametrically.

Rep. Ronny Jackson, R-Texas, on the health of the president as Biden turns 81 and Texas Gov. Abbott endorses Trump for the 2024 presidential election The former White House physician for Presidents Obama and Trump expressed concern Monday about President Biden's health and mental acuity as the president turns 81. Rep. Ronny Jackson, R-Texas, said on "FOX & Friends" that the growing concerns, including from the left, are valid. "I've been saying for quite some time now, when he was candidate Joe Biden, that I didn't think that he had the cognitive ability to do the job," said Jackson. Additionally, Jackson emphasized that Biden has "degenerated" over the last three years. "He's got these people that surround him that are inappropriately encouraging him to continue to run because it builds up who they are and what they do. But our border, our wars overseas, our economy, you know, it's just a disaster right now. And he just can't do the job. And it's just on display every day that he's not capable of doing this job anymore," Jackson warned.

By the time I sat down with Christopher Nolan in his posh hotel suite not far from the White House, I guessed that he was tired of Washington, D.C. The day before, he'd toured the Oval Office and had lunch on Capitol Hill. Later that night, I'd watched him receive an award from the Federation for American Scientists, an organization that counts Robert Oppenheimer, the subject of Nolan's most recent film, among its founders. He'd endured a joke, repeated too many times by Senate Majority Leader Chuck Schumer, about the subject of his next film--"It's another biopic: Schumer." The award was sitting on an end table next to Nolan, who was dressed in brown slacks, a gray vest, and a navy suit jacket--his Anglo-formality undimmed by decades spent living in Los Angeles. "It's heavy, and glass, and good for self-defense," he said of the award, while filling his teacup.

The world of gaming is being rocked by an AI controversy that could upend the multi-billion dollar industry. President Joe Biden's October 30 executive order on artificial intelligence is very broad and focuses a lot on safety but very little on freedom. It also provides a vague reference to minimal transparency by requiring AI companies to share results of red-team safety tests of their platforms which will eventually follow criteria of not-yet-developed standards by the National Institute of Standards and Technology. Many other government agencies, including Departments of Commerce, Treasury, Energy, Defense and Homeland Security are invoked with roles in AI regulation and safety standards. Indeed, there are only a few lonely agencies not directly invoked to participate in regulating AI.

The availability of representative datasets is an essential prerequisite for many successful artificial intelligence and machine learning models. However, in real life applications these models often encounter scenarios that are inadequately represented in the data used for training. There are various reasons for the absence of sufficient data, ranging from time and cost constraints to ethical considerations. As a consequence, the reliable usage of these models, especially in safety-critical applications, is still a tremendous challenge. Leveraging additional, already existing sources of knowledge is key to overcome the limitations of purely data-driven approaches. Knowledge augmented machine learning approaches offer the possibility of compensating for deficiencies, errors, or ambiguities in the data, thus increasing the generalization capability of the applied models. Even more, predictions that conform with knowledge are crucial for making trustworthy and safe decisions even in underrepresented scenarios. This work provides an overview of existing techniques and methods in the literature that combine data-driven models with existing knowledge. The identified approaches are structured according to the categories knowledge integration, extraction and conformity. In particular, we address the application of the presented methods in the field of autonomous driving.