Goto

Collaborating Authors

 Large Language Model


Copilot bug allows 'AI' to read confidential Outlook emails

PCWorld

PCWorld reports on a critical Microsoft Copilot bug (CW1226324) that allows the AI to scan and summarize confidential Outlook emails, bypassing privacy protections. This vulnerability affects Microsoft 365 accounts and compromises sensitive data like contracts and medical information stored in Sent and Drafts folders. Microsoft is rolling out a fix, but the timeline remains unclear, raising significant concerns about AI reliability and data privacy protection. For all its supposed intelligence, "AI" seems to make a lot of stupid mistakes--for example, scanning and summarizing emails marked "confidential" in Microsoft Outlook.



A Practitioner's Guide to Continual Multimodal Pretraining

Neural Information Processing Systems

However, practical model deployment often operates in the gap between these two limit cases, as real-world applications demand adaptation to specific subdomains, tasks or concepts -- spread over the entire, varying life cycle of a model.


Google DeepMind wants to know if chatbots are just virtue signaling

MIT Technology Review

Google DeepMind is calling for the moral behavior of large language models--such as what they do when called on to act as companions, therapists, medical advisors, and so on--to be scrutinized with the same kind of rigor as their ability to code or do math . As LLMs improve, people are asking them to play more and more sensitive roles in their lives. Agents are starting to take actions on people's behalf. LLMs may be able to influence human decision-making . And yet nobody knows how trustworthy this technology really is at such tasks. With coding and math, you have clear-cut, correct answers that you can check, William Isaac, a research scientist at Google DeepMind, told me when I met him and Julia Haas, a fellow research scientist at the firm, for an exclusive preview of their work, which is published in today. That's not the case for moral questions, which typically have a range of acceptable answers: "Morality is an important capability but hard to evaluate," says Isaac. "In the moral domain, there's no right and wrong," adds Haas.






Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approach Weiyu Ma

Neural Information Processing Systems

With the continued advancement of Large Language Models (LLMs) Agents in reasoning, planning, and decision-making, benchmarks have become crucial in evaluating these skills. However, there is a notable gap in benchmarks for real-time strategic decision-making. StarCraft II (SC2), with its complex and dynamic nature, serves as an ideal setting for such evaluations. To this end, we have developed TextStarCraft II, a specialized environment for assessing LLMs in real-time strategic scenarios within SC2. Addressing the limitations of traditional Chain of Thought (CoT) methods, we introduce the Chain of Summarization (CoS) method, enhancing LLMs' capabilities in rapid and effective decision-making. Our key experiments included: 1. LLM Evaluation: Tested 10 LLMs in TextStarCraft II, most of them defeating L V5 build-in AI, showcasing effective strategy skills.