Modern language models are powerful, but typically static after deployment. A major obstacle to building models that continually learn over time is catastrophic forgetting, where updating on new data erases previously acquired capabilities. Motivated by the intuition that mitigating forgetting is challenging because trainable parameters are shared across all tasks, we investigate whether sparse parameter updates can enable learning without catastrophic forgetting. We introduce sparse memory finetuning, leveraging memory layer models (Berges et al., 2024), which are sparsely updated by design. By updating only the memory slots that are highly activated by a new piece of knowledge relative to usage on pretraining data, we reduce interference between new knowledge and the model's existing capabilities. We evaluate learning and forgetting compared to full finetuning and parameter-efficient finetuning with LoRA on two question answering tasks. We find that sparse memory finetuning learns new knowledge while exhibiting substantially less forgetting: while NaturalQuestions F1 drops by 89% after full finetuning on new facts and 71% with LoRA, sparse memory finetuning yields only an 11% drop with the same level of new knowledge acquisition. Our results suggest sparsity in memory layers offers a promising path toward continual learning in large language models.

The California Department of Corrections and Rehabilitation is reviewing hundreds of state parole hearings to see if any inmates who were denied parole were rejected because of faulty drug tests. Nearly 6,000 drug tests in California prisons are believed to have yielded false positives between April and July last year, and attorneys for the Board of Parole are now conducting a review of inmate files to determine if any of them need to appear before the parole board again to be reconsidered, according to officials with CDCR. If any inmates were denied parole because of the faulty tests, they could be owed a new hearing before the parole board, said attorneys representing inmates affected by the defective drug tests. The review is already underway and will determine if "without the positive drug screening, there is sufficient evidence to support an incarcerated person's denial of parole," said CDCR spokesperson Emily Humpal in a statement. If there isn't enough evidence to support incarceration other than the drug test, a new hearing will be scheduled.

Fox News Flash top headlines are here. Check out what's clicking on Foxnews.com. New York's prison system unfairly punished more than 2,000 prisoners after tests of suspected contraband substances falsely tested positive for drugs, according to a report released Thursday. In hundreds of cases, the prisoners had committed no offense, but the flawed results were used to put them in solitary confinement, halt family visits, or cancel parole hearings. The report by Inspector General Lucy Lang found that state prison staff failed to confirm the test results with an outside lab.

Angelina Jolie was drug tested during production of 2001's "Lara Croft: Tomb Raider," according to a new book by the former CEO of Paramount Pictures. The Hollywood Reporter on Wednesday released an excerpt from an upcoming biography of the former studio head Sherry Lansing, "Leading Lady." Lansing stepped down as Paramount Pictures chairman and chief executive in 2005. The book claims when casting the then 24-year-old Jolie for "Lara Croft," Lansing had concerns about Jolie's stability after she received calls about the actress from her father Jon Voight and family friend Jane Fonda. The flick was filmed shortly before the actress wed Billy Bob Thornton.