Goto

Collaborating Authors

 grace


Navigating the Dual Facets: A Comprehensive Evaluation of Sequential Memory Editing in Large Language Models

arXiv.org Artificial Intelligence

Memory Editing (ME) has emerged as an efficient method to modify erroneous facts or inject new facts into Large Language Models (LLMs). Two mainstream ME methods exist: parameter-modifying ME and parameter-preserving ME (integrating extra modules while preserving original parameters). Regrettably, previous studies on ME evaluation have two critical limitations: (i) evaluating LLMs with single edit only, neglecting the need for continuous editing, and (ii) evaluations focusing solely on basic factual triples, overlooking broader LLM capabilities like logical reasoning and reading understanding. This study addresses these limitations with contributions threefold: (i) We explore how ME affects a wide range of fundamental capabilities of LLMs under sequential editing. Experimental results reveal an intriguing phenomenon: Most parameter-modifying ME consistently degrade performance across all tasks after a few sequential edits. In contrast, parameter-preserving ME effectively maintains LLMs' fundamental capabilities but struggles to accurately recall edited knowledge presented in a different format. (ii) We extend our evaluation to different editing settings, such as layers to edit, model size, instruction tuning, etc. Experimental findings indicate several strategies that can potentially mitigate the adverse effects of ME. (iii) We further explain why parameter-modifying ME damages LLMs from three dimensions: parameter changes after editing, language modeling capability, and the in-context learning capability. Our in-depth study advocates more careful use of ME in real-world scenarios.


2021.AI - The Enterprise AI Company

#artificialintelligence

Grace's comprehensive AI Governance framework offers you complete Governance, Risk & Compliance (GRC) for Data and AI, ensuring that you comply with the regulations and guidelines. Grace's Regulatory Excellence fully assures compliance for external Data and AI regulations, your internal code-of-conduct, best-practices, and guidelines. Ensure conformity and automate documentation, to ease the growing compliance burden for Data and AI and new disruptive technologies.


1587

AI Magazine

The Eighteenth National Conference on Artificial Intelligence (AAAI-2002) Robot Challenge is part of an annual series of robot challenges and competitions. It is intended to promote the development of robot systems that interact intelligently with humans in natural environments. The Challenge task calls for a robot to attend the AAAI conference, which includes registering for the conference and giving a talk about itself. In this article, we review the task requirements, introduce the robots that participated at AAAI-2002 and describe the strengths and weaknesses of their performance. The purpose of the challenge is to promote the development of robot systems that interact intelligently with humans in natural environments.


1584

AI Magazine

The Eleventh Annual AAAI Robot Competition and Exhibition was held at the National Conference on Artificial Intelligence in Edmonton, Alberta, Canada, in August 2002. This article describes each of the events that were held: Robot Challenge, Robot Exhibition, Robot Host, and Robot Rescue. Usually those attendees with names beginning AL are encouraged to line up behind one desk, and M-Z line up behind another. However, the 2002 National Conference on Artificial Intelligence included another desk: Robots! Some robots at the 2002 American Association for Artificial Intelligence (AAAI) Mobile Robot Competition and Exhibition actually registered for the conference on their own.


1602

AI Magazine

The main objectives of the challenge are to (1) provide a task that will demonstrate a high level of intelligence and autonomy for robots acting in a natural, peopled, dynamic environment; (2) stimulate state-of-the-art robotics research to address this task; and (3) use robot demonstrations to educate the public about the exciting and difficult challenges of robotics research. The challenge was designed as a problem that would probably need a decade to achieve adequately. When the challenge was designed, it was anticipated that no single research institution would have adequate resources to meet the challenge on its own. The challenge task is for a robot to participate in the American Association for Artificial Intelligence National Conference on Artificial Intelligence--the robot must find the registration booth and register, interacting with people as needed, then with a map in hand, find its way to a location in time to give a technical talk about itself. Ideally, the robot should be given no more information than any other participant arriving in a new city to attend a major technical conference.


2003 AAAI Robot Competition and Exhibition

AI Magazine

The Twelfth Annual American Association for Artificial Intelligence (AAAI) Robot Competition and Exhibition was held in Acapulco, Mexico, in conjunction with the Eighteenth International Joint Conference on Artificial Intelligence. The events included the Robot Host and Urban Search and Rescue competitions, the AAAI Robot Challenge, and the Robot Exhibition. In the Robot Host event, the robots had to act as mobile information servers and guides to the exhibit area of the conference. In the Urban Search and Rescue competition, teams attempted to find victims in a simulated disaster area using teleoperated, semiautonomous, and autonomous robots. The AAAI Robot Challenge is a noncompetitive event where the robots attempt to attend the conference by locating the registration booth, registering for the conference, and then giving a talk to an audience.


Can data shape the future of mental health support?

#artificialintelligence

If you're experiencing a mental health issue, one of the people you probably least want to speak to about it is your employer. Disclosing depression or anxiety has long been seen as the last workplace taboo, for fear of repercussions. This is despite the existence of the Equality Act 2010, which protects employees with physical and mental disabilities from discrimination. But just over a third of workers with a mental health condition discuss it with their employer, according to a survey of 1,388 employees carried out by Willis PMI Group, one of the UK's largest providers of employee healthcare and risk management services. The research found that 30% of respondents were concerned that they wouldn't receive adequate support, 28% believed their employer wouldn't understand, and 23% feared that disclosing it would lead to management thinking less of them.


Virtual Humans for Learning

AI Magazine

Virtual humans are computer-generated characters designed to look and behave like real people. Studies have shown that virtual humans can mimic many of the social effects that one finds in human-human interactions such as creating rapport, and people respond to virtual humans in ways that are similar to how they respond to real people. We believe that virtual humans represent a new metaphor for interacting with computers, one in which working with a computer becomes much like interacting with a person and this can bring social elements to the interaction that are not easily supported with conventional interfaces. We present two systems that embody these ideas. The first, the Twins are virtual docents in the Museum of Science, Boston, designed to engage visitors and raise their awareness and knowledge of science. The second SimCoach, uses an empathetic virtual human to provide veterans and their families with information about PTSD and depression.


GRACE: An Autonomous Robot for the AAAI Robot Challenge

AI Magazine

In an attempt to solve as much of the AAAI Robot Challenge as possible, five research institutions representing academia, industry, and government integrated their research into a single robot named GRACE. This article describes this first-year effort by the GRACE team, including not only the various techniques each participant brought to GRACE but also the difficult integration effort itself.


The AAAI-2002 Robot Challenge

AI Magazine

The Eighteenth National Conference on Artificial Intelligence (AAAI-2002) Robot Challenge is part of an annual series of robot challenges and competitions. It is intended to promote the development of robot systems that interact intelligently with humans in natural environments. The Challenge task calls for a robot to attend the AAAI conference, which includes registering for the conference and giving a talk about itself. In this article, we review the task requirements, introduce the robots that participated at AAAI-2002 and describe the strengths and weaknesses of their performance.