Government
Conserve-Update-Revise to Cure Generalization and Robustness Trade-off in Adversarial Training
Gowda, Shruthi, Zonooz, Bahram, Arani, Elahe
Adversarial training improves the robustness of neural networks against adversarial attacks, albeit at the expense of the trade-off between standard and robust generalization. To unveil the underlying factors driving this phenomenon, we examine the layer-wise learning capabilities of neural networks during the transition from a standard to an adversarial setting. Our empirical findings demonstrate that selectively updating specific layers while preserving others can substantially enhance the network's learning capacity. We therefore propose CURE, a novel training framework that leverages a gradient prominence criterion to perform selective conservation, updating, and revision of weights. Importantly, CURE is designed to be dataset-and architecture-agnostic, ensuring its applicability across various scenarios. It effectively tackles both memorization and overfitting issues, thus enhancing the trade-off between robustness and generalization and additionally, this training approach also aids in mitigating "robust overfitting". Furthermore, our study provides valuable insights into the mechanisms of selective adversarial training and offers a promising avenue for future research. The susceptibility of deep neural networks (DNNs) to adversarial attacks (Szegedy et al., 2014; Goodfellow et al., 2015) continues to present a substantial challenge in the field. Adversarial training has emerged as a promising strategy to enhance the robustness of DNNs against adversarial attacks (Madry et al., 2018; Zhang et al., 2019; Tramèr et al., 2018; Wang et al., 2019). However, transitioning from standard training with natural images to adversarial training introduces distinct behavior patterns. Despite the benefits of adversarial training in improving robustness, it often results in compromised performance on clean images, creating a noticeable trade-off between standard and adversarial generalization (Raghunathan et al., 2019). Another intriguing observation is that, in contrast to the standard setting, longer durations of adversarial training can paradoxically lead to reduced test performance. This generalization gap in robustness between training and testing data, commonly referred to as robust overfitting (Rice et al., 2020), is prevalent in adversarial training. Therefore, it is imperative to gain a deeper understanding of the underlying factors driving these behaviors to advance the development of reliable and trustworthy AI systems. Few studies have attempted to understand learning behavior in an adversarial setting.
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly
Yao, Yifan, Duan, Jinhao, Xu, Kaidi, Cai, Yuanfang, Sun, Zhibo, Zhang, Yue
Large Language Models (LLMs), such as ChatGPT and Bard, have revolutionized natural language understanding and generation. They possess deep language comprehension, human-like text generation capabilities, contextual awareness, and robust problem-solving skills, making them invaluable in various domains (e.g., search engines, customer support, translation). In the meantime, LLMs have also gained traction in the security community, revealing security vulnerabilities and showcasing their potential in security-related tasks. This paper explores the intersection of LLMs with security and privacy. Specifically, we investigate how LLMs positively impact security and privacy, potential risks and threats associated with their use, and inherent vulnerabilities within LLMs. Through a comprehensive literature review, the paper categorizes the papers into "The Good" (beneficial LLM applications), "The Bad" (offensive applications), and "The Ugly" (vulnerabilities of LLMs and their defenses). We have some interesting findings. For example, LLMs have proven to enhance code security (code vulnerability detection) and data privacy (data confidentiality protection), outperforming traditional methods. However, they can also be harnessed for various attacks (particularly user-level attacks) due to their human-like reasoning abilities. We have identified areas that require further research efforts. For example, Research on model and parameter extraction attacks is limited and often theoretical, hindered by LLM parameter scale and confidentiality. Safe instruction tuning, a recent development, requires more exploration. We hope that our work can shed light on the LLMs' potential to both bolster and jeopardize cybersecurity.
Leveraging Generative AI for Clinical Evidence Summarization Needs to Ensure Trustworthiness
Zhang, Gongbo, Jin, Qiao, McInerney, Denis Jered, Chen, Yong, Wang, Fei, Cole, Curtis L., Yang, Qian, Wang, Yanshan, Malin, Bradley A., Peleg, Mor, Wallace, Byron C., Lu, Zhiyong, Weng, Chunhua, Peng, Yifan
Evidence-based medicine promises to improve the quality of healthcare by empowering medical decisions and practices with the best available evidence. The rapid growth of medical evidence, which can be obtained from various sources, poses a challenge in collecting, appraising, and synthesizing the evidential information. Recent advancements in generative AI, exemplified by large language models, hold promise in facilitating the arduous task. However, developing accountable, fair, and inclusive models remains a complicated undertaking. In this perspective, we discuss the trustworthiness of generative AI in the context of automated summarization of medical evidence.
Big-Name Targets Push Midnight Blizzard Hacking Spree Back Into the Limelight
Microsoft and Hewlett-Packard Enterprise (HPE) both recently disclosed that they suffered corporate email breaches at the hands of Russia's "Midnight Blizzard" hackers. The group, which is tied to the Kremlin's SVR foreign intelligence, is specifically linked to SVR's APT 29 Cozy Bear, the gang that meddled in the United States 2016 presidential election, has conducted aggressive government and corporate espionage around the world for years, and was behind the infamous 2021 SolarWinds supply chain attack. While both HP and Microsoft's breaches came to light within days of each other, the situation mainly illustrates the ongoing reality of Midnight Blizzard's international espionage activities and the lengths it will go to to find weaknesses in organizations' digital defenses. "We shouldn't be surprised that Russian intelligence-backed threat actors, and SVR in particular, are targeting tech companies like Microsoft and HPE. With organizations that size, it would be a much bigger surprise to learn they weren't," says Jake Williams, a former US National Security Agency hacker and current faculty member at the Institute for Applied Network Security.
Japan's SLIM lunar spacecraft landed upside down on the moon
Shortly after Japan's space agency became the fifth country to land a spacecraft on the surface of the moon, its scientists discovered the Smart Lander for Investigating Moon (SLIM) unfortunately touched down upside down. The Japan Aerospace Exploration Agency (JAXA) said that the SLIM landed on the lunar surface on January 20 but it knew it might have bigger problems due to an issue with power generation. Just hours after making landfall, JAXA expected the power to run out, before it ultimately did. SLIM met the moon's surface about 55 meters east of the original target landing site, JAXA said. The agency did get all of the technical information related to its navigation prior to landing and ultimately becoming stationary on the lunar surface.
Robot Car Crash Investigation Concludes GM's Cruise Didn't Disclose Key Information
A law firm hired by the General Motors' self-driving subsidiary Cruise to investigate the company's response to a gruesome San Francisco crash last year found that the company failed to fully disclose disturbing details to regulators, the tech company said today in a blog post. The incident in October led California regulators to suspend Cruise's license to operate driverless vehicles in San Francisco. The new report by law firm Quinn Emanuel says that Cruise failed to tell California's Department of Motor Vehicles that after striking a pedestrian knocked into its path by a human-driven vehicle, the autonomous car pulled out of traffic--dragging her some 20 feet. Cruise said it had accepted the firm, Quinn Emanuel's, version of events, as well as its recommendations. The investigators found that when Cruise played a video of the crash taken from its autonomous vehicle for government officials, it did not "verbally point out" the vehicle's pullover maneuver.
Americans hit with 78 BILLION robocall scams each year, new report reveals after AI-cloned voice of Joe Biden urged New Hampshire Democrats not to vote in primary
A new report reveals that Robocall scam are on the rise in America amid the advancement of AI that can clone voices - even that of the US President Joe Biden. A fake recorded message impersonating Biden was unleashed in New Hampshire this week, which urged Democrats now to vote in the primary. 'Voting this Tuesday only enables the Republicans in their quest to elect Donald Trump again. Your vote makes a difference in November, not this Tuesday,' victims heard on the phone. The malicious campaign highlights the dangers of technology that is running rampant in the US - Americans are hit with 78 billion robocalls and 225 billion robotexts per year - a more than 50 percent jump from 2021.
The FTC is investigating Microsoft, Amazon and Alphabet's investments into AI startups
The Federal Trade Commission is launching an inquiry into massive investments made by Microsoft, Amazon and Alphabet into generative AI startups OpenAI and Anthropic, the agency announced on Thursday. The FTC said that it had issued "compulsory orders" to the companies and would scrutinize their relationships with AI startups to understand their impact on competition. "History shows that new technologies can create new markets and healthy competition," FTC Chair Lina Khan said in a statement. "As companies race to develop and monetize AI, we must guard against tactics that foreclose this opportunity. Our study will shed light on whether investments and partnerships pursued by dominant companies risk distorting innovation and undermining fair competition."
Federal Trade Commission scrutinizes Big Tech's AI deals
Under the Biden administration, federal regulators have stepped up their scrutiny of Big Tech companies' acquisitions of smaller rivals, bringing lengthy and costly legal challenges against Meta's acquisition of the virtual reality company Within and Microsoft's purchase of the game maker Activision. In the age of generative AI, Silicon Valley giants have to date sidestepped such legal obstacles by instead funneling investments into younger AI companies and striking deals to ensure those start-ups are giving preference to their computing services.