relearn
ReLearn: Unlearning via Learning for Large Language Models
Xu, Haoming, Zhao, Ningyuan, Yang, Liming, Zhao, Sendong, Deng, Shumin, Wang, Mengru, Hooi, Bryan, Oo, Nay, Chen, Huajun, Zhang, Ningyu
Current unlearning methods for large language models usually rely on reverse optimization to reduce target token probabilities. However, this paradigm disrupts the subsequent tokens prediction, degrading model performance and linguistic coherence. Moreover, existing evaluation metrics overemphasize contextual forgetting while inadequately assessing response fluency and relevance. To address these challenges, we propose ReLearn, a data augmentation and fine-tuning pipeline for effective unlearning, along with a comprehensive evaluation framework. This framework introduces Knowledge Forgetting Rate (KFR) and Knowledge Retention Rate (KRR) to measure knowledge-level preservation, and Linguistic Score (LS) to evaluate generation quality. Our experiments show that ReLearn successfully achieves targeted forgetting while preserving high-quality output. Through mechanistic analysis, we further demonstrate how reverse optimization disrupts coherent text generation, while ReLearn preserves this essential capability. Code is available at https://github.com/zjunlp/unlearn.
- Law (1.00)
- Information Technology > Security & Privacy (1.00)
- Health & Medicine > Therapeutic Area > Neurology (0.94)
Tracking Time-varying Graphical Structure
Structure learning algorithms for graphical models have focused almost exclusively on stable environments in which the underlying generative process does not change; that is, they assume that the generating model is globally stationary. In real-world environments, however, such changes often occur without warning or signal. Real-world data often come from generating models that are only locally stationary. In this paper, we present LoSST, a novel, heuristic structure learning algorithm that tracks changes in graphical model structure or parameters in a dynamic, real-time manner. We show by simulation that the algorithm performs comparably to batch-mode learning when the generating graphical structure is globally stationary, and significantly better when it is only locally stationary.
- North America > United States > Pennsylvania > Allegheny County > Pittsburgh (0.28)
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.28)
- Asia > India (0.04)
How Artificial Intelligence Is Improving the Energy Efficiency of Buildings
A lot of energy is consumed by buildings. In fact, the Alliance to Save Energy, a nonprofit energy efficiency advocacy group, says buildings account for about 40% of all U.S. energy consumption and a similar proportion of greenhouse gas emissions. Some estimates suggest about 45% of the energy used in commercial buildings is consumed by heating, ventilation, and air conditioning (HVAC) systems, of which, as much as 30% is often wasted. Most power companies these days have energy efficiency programs that help customers identify waste and implement energy-saving measures, but there are also non-utility providers working on solutions. Montreal, Canada–based BrainBox AI is one of them. It's using artificial intelligence (AI) to significantly reduce energy consumption in buildings.
- Energy > Power Industry (0.77)
- Construction & Engineering > HVAC (0.77)
Opinion
In a way I was wrong, and in a way I was right. A.I. programs are capable of mimicking and even surpassing human brains in many tasks. But if A.I. allows us to truly understand ourselves, it will be because it liberates us from the mechanical drudgery of routine tasks and allows us to focus on our humanity and the compassionate connections between us. We already know that many of the jobs that are being replaced will not return, because A.I. can do them much better than people at essentially zero cost. This will generate tremendous economic value but will also result in unprecedented job displacement.
- North America > United States > California (0.08)
- Asia > China (0.08)
There is Strength in Numbers: Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training
Stacey, Joe, Minervini, Pasquale, Dubossarsky, Haim, Riedel, Sebastian, Rocktäschel, Tim
Natural Language Inference (NLI) datasets contain annotation artefacts resulting in spurious correlations between the natural language utterances and their respective entailment classes. These artefacts are exploited by neural networks even when only considering the hypothesis and ignoring the premise, leading to unwanted biases. Previous work proposed tackling this problem via adversarial training, but this leads to learned sentence representations that still suffer from the same biases. As a solution, we propose using an ensemble of adversaries during the training, encouraging the model to jointly decrease the accuracy of these different adversaries while fitting the data. We show that using an ensemble of adversaries can prevent the bias from being relearned after the model training is completed, further improving how well the model generalises to different NLI datasets. In particular, these models outperformed previous approaches when tested on 12 different NLI datasets not used in the model training. Finally, the optimal number of adversarial classifiers depends on the dimensionality of the sentence representations, with larger dimensional representations benefiting when trained with a greater number of adversaries.
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.04)
- Asia > Middle East > Jordan (0.04)
- Research Report > New Finding (0.68)
- Research Report > Experimental Study (0.47)
Artificial Intelligence: A Need of Modern 'Intelligent' Education - Thrive Global
Artificial intelligence is influencing the future of virtually every industry and every human being. It has acted as the main driver of emerging technologies like big data, robotics, and IoT, and it will continue to act as a technological innovator for the near future. According to tech experts, artificial intelligence (AI) has the potential to transform the world. However, those same experts do not agree on what kind of effect that transformation will have on the average person. Some believe that humans will be much better off in the hands of advanced AI systems, while others think it will lead to our inevitable downfall.
- Health & Medicine > Therapeutic Area > Psychiatry/Psychology (1.00)
- Education (1.00)
How AI will redefine creative careers and job roles
In recent years, we've seen many digitally led organisations recruit developers, experience designers (including digital and motion graphics designers) to form fully functional design teams. However, the advent of AI has meant art directors, copywriters can now work closely with designers and developers to use AI tools and bring their ideas to life through critical assessment. To stay competitive, today's creative needs to be well versed in a range of topics while still having a fluid approach to new ideas. As a future fit professional, your ability to critically self-assess your talents, skills, deconstructing and reconstructing them, in the context of your organisation, team and the wider market, will keep you ahead of the competition.
?utm_content=buffer93ca9&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer
In recent years, we've seen many digitally led organisations recruit developers, experience designers (including digital and motion graphics designers) to form fully functional design teams. However, the advent of AI has meant art directors, copywriters can now work closely with designers and developers to use AI tools and bring their ideas to life through critical assessment. To stay competitive, today's creative needs to be well versed in a range of topics while still having a fluid approach to new ideas. As a future fit professional, your ability to critically self-assess your talents, skills, deconstructing and reconstructing them, in the context of your organisation, team and the wider market, will keep you ahead of the competition.
Tracking Time-varying Graphical Structure
Kummerfeld, Erich, Danks, David
Structure learning algorithms for graphical models have focused almost exclusively on stable environments in which the underlying generative process does not change; that is, they assume that the generating model is globally stationary. In real-world environments, however, such changes often occur without warning or signal. Real-world data often come from generating models that are only locally stationary. In this paper, we present LoSST, a novel, heuristic structure learning algorithm that tracks changes in graphical model structure or parameters in a dynamic, real-time manner. We show by simulation that the algorithm performs comparably to batch-mode learning when the generating graphical structure is globally stationary, and significantly better when it is only locally stationary.
- North America > United States > Pennsylvania > Allegheny County > Pittsburgh (0.28)
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.28)
- Asia > India (0.04)