Large Language Models (LLMs) offer extensive knowledge across various domains, but they may inadvertently memorize sensitive, unauthorized, or malicious data, such as personal information in the medical and financial sectors. Machine unlearning methods aim to remove specific information from models after training to address this. However, current approaches require additional model training or struggle to effectively erase particular data points and their associated context due to LLMs' complex, dense, and continuous nature. In this study, we propose a novel amortized unlearning approach using codebook features and Sparse Autoencoders (SAEs). By leveraging a bottleneck to decompose the activation space and regulate information flow, our method efficiently unlearns targeted information while preserving the model's performance on unrelated data. To the best of our knowledge, this is the first work that successfully enables unlearning specific topics with contextual relevance in an LLM, marking a significant step towards real-world applications of machine unlearning. Large language Models (LLMs) have been widely used in various applications, generating text responses that attempt to create the equivalent of human conversations OpenAI et al. (2024). These models leverage vast scientific literature to facilitate and accelerate interdisciplinary research Taylor et al. (2022) while drawing upon large datasets of human-generated content to provide professional advice. However, in many cases, such data is a double-edged sword. Including personal information or sensitive scientific knowledge can be beneficial or, conversely, harmful. For instance, Soice et al. (2023) discusses how LLMs, when used by non-experts, can enable the creation of biological agents, posing both potential benefits and significant risks.

Julien Simon is a principal AI and machine learning developer advocate. He focuses on helping developers and enterprises to bring their ideas to life. He frequently speaks at conferences and blogs on AWS blogs and on Medium. Prior to joining AWS, Julien served for 10 years as CTO/VP of engineering in top-tier web start-ups where he led large software and ops teams in charge of thousands of servers worldwide. In the process, he fought his way through a wide range of technical, business, and procurement issues, which helped him gain a deep understanding of physical infrastructure, its limitations, and how cloud computing can help.

One-stop-shop to get information into the history, development and potential of GPT-3. Julien Lauret's article is a comprehensive summary of the journey taken so far to create GPT-3. Julien has managed to summarize years of development and introductions of methodology and techniques to model language and solve natural language processing into several small, concise paragraphs. As well as providing the reader with a background of GPT-3, Julien also gives a somewhat diplomatic answer to the question as to whether GPT-3 is AGI. His response truly reflects the nature of the question itself, in that the question is subjected to the definition of intelligence by whoever poses the question.

The interest for bots is not going to slow down: a study shows that 80% of businesses want to start using chatbots in the next two years. To implement the technology properly, companies should be aware of the real potential and limitations. Like any new technology, chatbots arouse hopes and fears. Now that companies are integrating it into their strategies, we can better understand how it is beneficial for both companies and customers alike. To help you implement your chatbot and know what to expect, we have gathered eight of the most common myths about chatbots and have called on industry experts to debunk these myths.

Our Artificial Intelligence app, Hugging Face, has been running smoothly following a big influx of new users. It's a normal day, and I'm looking over activity readouts when suddenly the app grinds to a complete halt. Thousands of teens chatting with their AI friends are getting nothing but silence in return. I pull up Slack, and ask the tech team if we are down. Julien, my co-founder and the CTO of Hugging Face, looks over the brains of our AIs, and comes up with nothing out of the ordinary.