Government
Measuring the Robustness of Reference-Free Dialogue Evaluation Systems
Vasselli, Justin, Nohejl, Adam, Watanabe, Taro
Advancements in dialogue systems powered by large language models (LLMs) have outpaced the development of reliable evaluation metrics, particularly for diverse and creative responses. We present a benchmark for evaluating the robustness of reference-free dialogue metrics against four categories of adversarial attacks: speaker tag prefixes, static responses, ungrammatical responses, and repeated conversational context. We analyze metrics such as DialogRPT, UniEval, and PromptEval -- a prompt-based method leveraging LLMs -- across grounded and ungrounded datasets. By examining both their correlation with human judgment and susceptibility to adversarial attacks, we find that these two axes are not always aligned; metrics that appear to be equivalent when judged by traditional benchmarks may, in fact, vary in their scores of adversarial responses. These findings motivate the development of nuanced evaluation frameworks to address real-world dialogue challenges.
Scam Detection for Ethereum Smart Contracts: Leveraging Graph Representation Learning for Secure Blockchain
Due to the increasing abuse of fraudulent activities that result in significant financial and reputational harm, Ethereum smart contracts face a significant problem in detecting fraud. Existing monitoring methods typically rely on lease code analysis or physically extracted features, which suffer from scalability and adaptability limitations. In this study, we use graph representation learning to observe purchase trends and find fraudulent deals. We can achieve powerful categorisation performance by using innovative machine learning versions and transforming Ethereum invoice data into graph structures. Our method addresses label imbalance through SMOTE-ENN techniques and evaluates models like Multi-Layer Perceptron ( MLP ) and Graph Convolutional Networks ( GCN). Experimental results show that the MLP type surpasses the GCN in this environment, with domain-specific assessments closely aligned with real-world assessments. This study provides a scalable and efficient way to improve Ethereum's ecosystem's confidence and security.
Technology as uncharted territory: Contextual integrity and the notion of AI as new ethical ground
Recent research illustrates how AI can be developed and deployed in a manner detached from the concrete social context of application. By abstracting from the contexts of AI application, practitioners also disengage from the distinct normative structures that govern them. Building upon Helen Nissenbaum's framework of contextual integrity, I illustrate how disregard for contextual norms can threaten the integrity of a context with often decisive ethical implications. I argue that efforts to promote responsible and ethical AI can inadvertently contribute to and seemingly legitimize this disregard for established contextual norms. Echoing a persistent undercurrent in technology ethics of understanding emerging technologies as uncharted moral territory, certain approaches to AI ethics can promote a notion of AI as a novel and distinct realm for ethical deliberation, norm setting, and virtue cultivation. This narrative of AI as new ethical ground, however, can come at the expense of practitioners, policymakers and ethicists engaging with already established norms and virtues that were gradually cultivated to promote successful and responsible practice within concrete social contexts. In response, I question the current narrow prioritization in AI ethics of moral innovation over moral preservation. Engaging also with emerging foundation models, I advocate for a moderately conservative approach to the ethics of AI that prioritizes the responsible and considered integration of AI within established social contexts and their respective normative structures.
UK can be 'AI sweet spot': Starmer's tech minister on regulation, Musk, and free speech
With the NHS still struggling, a prisons crisis still teetering and Britain's borrowing costs soaring, there are few easy jobs going in Keir Starmer's cabinet at present. But even in such difficult times, the task of convincing Silicon Valley's finest to help make Britain a leader in the artificial intelligence (AI) revolution โ all while one leading tech boss uses the Labour government as a regular punching bag and others ostentatiously move closer to Donald Trump โ is among the most challenging. This is the mission that has fallen to Peter Kyle, the science and technology secretary, who has become an important figure in Starmer's cabinet. If balancing the concerns over online free speech, AI's impact on the climate crisis and the threat it poses to wiping out humanity are not enough, the economic headwinds Britain is now experiencing makes the launch this week of the government's AI action plan even more important. And Kyle is worried Britain could miss the boat.
Tech giants told UK online safety laws 'not up for negotiation'
Britain's new laws to boost safety and tackle hate speech online are "not up for negotiation", a senior government minister has warned, after Meta founder Mark Zuckerberg vowed to join Donald Trump to pressure countries they regard as "censoring" content. In an interview with the Observer, Peter Kyle, the technology secretary, said that the recent laws designed to make online platforms safer for children and vulnerable people would never be diluted to help the government woo big tech companies to the UK in its defining pursuit for economic growth. His comments come as Keir Starmer prepares a major big tech charm offensive this week in which he will pitch the UK as the "sweet spot" for the development of artificial intelligence (AI) technology. However, the prime minister will do so with his government facing constant and wild attacks from Elon Musk, one of Silicon Valley's most prominent figures and a leading Trump supporter. Zuckerberg also used a wide-ranging statement last week to reveal he was ditching "politically biased" factcheckers and reducing restrictions on topics such as immigration and gender on Meta's platforms, including Facebook, Instagram and Threads.
The top 3 factors heightening the risk of terror attacks on the homeland
As a former military intelligence officer, serving in the Defense Intelligence Agency (DIA), I tracked foreign threats to the U.S. homeland, identifying adversaries' plans, intentions and capabilities that could harm Americans. I predicted Russia's invasion of Ukraine more than a year before it took place. In March, in my Fox News Digital article titled "Ignore FBI director's urgent warning about terrorist threats at our own peril," I predicted terrorist attacks striking inside the U.S. homeland, the kind that took place on New Year's Day in New Orleans and in Las Vegas. Here are the top three reasons why we will likely face more terrorism in America this year. This time, it will be something we haven't seen before.
Fox News AI Newsletter: Tech leaders' message to Biden
Nvidia is developing real-world robots that are equipped with artificial intelligence capabilities. PUSH BACK: The new rule, which industry leaders say could come as early as the end of this week, effectively seeks to shore up the U.S. economy and national security efforts by adding new restrictions on how many U.S.-made artifical intelligence products can be deployed across the globe. Jensen Huang, co-founder and chief executive officer of Nvidia Corp., speaks during the Nvidia GPU Technology Conference (GTC) in San Jose, Calif., on Monday, March 18, 2024. 'UTTERLY UNTRUE': Open AI CEO Sam Altman on Tuesday responded to a lawsuit in which his sister accused him of sexually abusing her for nearly a decade. Altman, along with his mother and two brothers, issued a joint statement denying the claims of his sister, Ann Altman.
'Incredibly dangerous': More unauthorized drones fly above Palisades fire
Multiple unauthorized drones flew above the Palisades fire Friday afternoon, forcing firefighting aircraft to leave the area for safety and angering those working on the front lines, authorities said. These sightings came just a day after a drone collided with a Super Scooper fixed-wing aircraft, grounding the plane for several days of repairs and reducing the number of aircraft available to fight the fire. "This is not just harmless fun. This is incredibly dangerous," said Chris Thomas, public information officer for the Palisades fire. "Seriously, what if that plane had gone down? It could have taken out a row of homes. It could have taken out a school."
Large Language Models, Knowledge Graphs and Search Engines: A Crossroads for Answering Users' Questions
Hogan, Aidan, Dong, Xin Luna, Vrandeฤiฤ, Denny, Weikum, Gerhard
Much has been discussed about how Large Language Models, Knowledge Graphs and Search Engines can be combined in a synergistic manner. A dimension largely absent from current academic discourse is the user perspective. In particular, there remain many open questions regarding how best to address the diverse information needs of users, incorporating varying facets and levels of difficulty. This paper introduces a taxonomy of user information needs, which guides us to study the pros, cons and possible synergies of Large Language Models, Knowledge Graphs and Search Engines. From this study, we derive a roadmap for future research.
Understanding and Mitigating Membership Inference Risks of Neural Ordinary Differential Equations
Hong, Sanghyun, Wu, Fan, Gruber, Anthony, Lee, Kookjin
Neural ordinary differential equations (NODEs) are an emerging paradigm in scientific computing for modeling dynamical systems. By accurately learning underlying dynamics in data in the form of differential equations, NODEs have been widely adopted in various domains, such as healthcare, finance, computer vision, and language modeling. However, there remains a limited understanding of the privacy implications of these fundamentally different models, particularly with regard to their membership inference risks. In this work, we study the membership inference risks associated with NODEs. We first comprehensively evaluate NODEs against membership inference attacks. We show that NODEs are twice as resistant to these privacy attacks compared to conventional feedforward models such as ResNets. By analyzing the variance in membership risks across different NODE models, we identify the factors that contribute to their lower risks. We then demonstrate, both theoretically and empirically, that membership inference risks can be further mitigated by utilizing a stochastic variant of NODEs: Neural stochastic differential equations (NSDEs). We show that NSDEs are differentially-private (DP) learners that provide the same provable privacy guarantees as DP-SGD, the de-facto mechanism for training private models. NSDEs are also effective in mitigating existing membership inference attacks, demonstrating risks comparable to private models trained with DP-SGD while offering an improved privacy-utility trade-off. Moreover, we propose a drop-in-replacement strategy that efficiently integrates NSDEs into conventional feedforward models to enhance their privacy.