Goto

Collaborating Authors

 Government


What You See Is Not Always What You Get: An Empirical Study of Code Comprehension by Large Language Models

arXiv.org Artificial Intelligence

Recent studies have demonstrated outstanding capabilities of large language models (LLMs) in software engineering domain, covering numerous tasks such as code generation and comprehension. While the benefit of LLMs for coding task is well noted, it is perceived that LLMs are vulnerable to adversarial attacks. In this paper, we study the specific LLM vulnerability to imperceptible character attacks, a type of prompt-injection attack that uses special characters to befuddle an LLM whilst keeping the attack hidden to human eyes. We devise four categories of attacks and investigate their effects on the performance outcomes of tasks relating to code analysis and code comprehension. Two generations of ChatGPT are included to evaluate the impact of advancements made to contemporary models. Our experimental design consisted of comparing perturbed and unperturbed code snippets and evaluating two performance outcomes, which are model confidence using log probabilities of response, and correctness of response. We conclude that earlier version of ChatGPT exhibits a strong negative linear correlation between the amount of perturbation and the performance outcomes, while the recent ChatGPT presents a strong negative correlation between the presence of perturbation and performance outcomes, but no valid correlational relationship between perturbation budget and performance outcomes. We anticipate this work contributes to an in-depth understanding of leveraging LLMs for coding tasks. It is suggested future research should delve into how to create LLMs that can return a correct response even if the prompt exhibits perturbations.


Thinking Fast and Laterally: Multi-Agentic Approach for Reasoning about Uncertain Emerging Events

arXiv.org Artificial Intelligence

This paper introduces lateral thinking to implement System-2 reasoning capabilities in AI systems, focusing on anticipatory and causal reasoning under uncertainty. We present a framework for systematic generation and modeling of lateral thinking queries and evaluation datasets. We introduce Streaming Agentic Lateral Thinking (SALT), a multi-agent framework designed to process complex, low-specificity queries in streaming data environments. SALT implements lateral thinking-inspired System-2 reasoning through a dynamic communication structure between specialized agents. Our key insight is that lateral information flow across long-distance agent interactions, combined with fine-grained belief management, yields richer information contexts and enhanced reasoning. Preliminary quantitative and qualitative evaluations indicate SALT's potential to outperform single-agent systems in handling complex lateral reasoning tasks in a streaming environment.


Compliance Requirements in Research

Communications of the ACM

Scientific research has traditionally been conducted in a culture of openness. However, increasingly in the last decade, various concerns have resulted in more regulations about keeping research data and processes secure from accidental corruption and loss and--more importantly--from theft. While the data is the primary asset of value that needs to be protected, various flavors of data are part of workflows that can be complex and require special infrastructure to support. Examples include raw data from measurement devices and sensors, intermediate data in the form of lab notes, computer software and simulation datasets, and final reports and publications. To keep data secure, laboratory spaces, instruments, computer systems, and data networks are in scope for protection.


Chinese regulators are investigating NVIDIA for potential antitrust violations

Engadget

NVIDIA, graphics chip maker and recent backbone of the AI industry, is under investigation by Chinese regulators over potential antitrust violations, The New York Times reports. The concerns center on the acquisition of Mellanox Technologies, a computer networking company NVIDIA bought in 2020. As part of the conditions of that acquisition, Chinese regulators required NVIDIA to "provide information about new [Mellanox] products to rivals within 90 days of making them available to NVIDIA," Bloomberg writes. China's State Administration for Market Regulation is kicking off its investigation because it believes that those terms were violated. This wouldn't be the first time NVIDIA has been investigated for monopolistic behavior โ€“ The US Department of Justice reportedly launched its own antitrust investigation into NVIDIA in September 2024 โ€“ but it has a different flavor in the context of the escalating trade war between the US and China.


Mysterious drones are 'changing time' on clocks in New Jersey as locals fear they're being targeted by UFOs

Daily Mail - Science & tech

As waves of loud, car-sized mystery drones continue to buzz over New Jersey, one family reported that the craft changed time on their car's clock. The family of Morris County locals said they were following one of these seemingly terrestrial UFOs in their vehicle, only to experience the odd effect on their car's electronics as the unexplained craft'hovered above them.' 'The clock in their car changed time,' according to one Fox News reporter who spoke to the unnamed family. 'They say the clock went back to normal after they drove off.' While local law enforcement in Morris County has issued a statement asserting that'there is no known threat to public safety' at this time -- the Federal Aviation Administration (FAA) has issued a ban on drone flights over sensitive areas in state.


Mysterious SUV-sized drones may have blocked medical helicopter

Popular Science

Residents and law enforcement officials are reporting numerous large, unidentified drones flying at night over New Jersey, New York, and Pennsylvania. Some of the fixed-wing devices are estimated to be roughly four-feet-wide, while others are believed to be as large as a car. And while officials including New Jersey governor Phil Murphy have stressed there is no evidence suggesting the drones pose a threat to public safety, at least one related incident may have delayed medevac transport of a seriously injured car wreck victim. As The New York Times noted over the weekend, sightings date as far back as November, and have occurred over residential areas, highways, railroads, reservoirs, and power lines. In most instances, the loud, blinking drones appear to be "significantly larger" than standard drones piloted by hobbyists. At least two events prompted the Federal Aviation Administration to temporarily ban drones at Donald Trump's Bedminster National Golf Club and Picatinny Arsenal, a 6,400-square-acre military research and manufacturing facility in Morris County, New Jersey.


Pentagon announces new counter-drone strategy as unmanned attacks on US interests skyrocket

FOX News

Fox News' Stephanie Bennett reports the latest on the unidentified drones from London. The Pentagon unveiled a new counter-drone strategy after a spate of incursions near U.S. bases prompted concerns over a lack of an action plan for the increasing threat of unmanned aerial vehicles. Though much of the strategy remains classified, Defense Secretary Lloyd Austin will implement a new counter-drone office within the Pentagon โ€“ Joint Counter-Small UAS Office โ€“ and a new Warfighter Senior Integration Group, according to a new memo. The Pentagon will also begin work on a second Replicator initiative, but it will be up to the incoming Trump administration to decide whether to fund this plan. The first Replicator initiative worked to field inexpensive, dispensable drones to thwart drone attacks by adversarial groups across the Middle East and elsewhere.


Houthis claim attack on central Israel in response to Gaza 'massacres'

Al Jazeera

Yemen's Houthi group says it has carried out a drone attack in central Israel's Tel Aviv area in "a specific military operation" in support of Palestinians in Gaza. The Houthis said in a statement on Monday that their forces struck "a sensitive target of the Israeli enemy". An Israeli military statement said a drone hit a building in the city of Yavne after air defence systems failed to detect it and an investigation into the failure is under way. The Houthis said the operation "achieved its objective" without providing details. No injuries were reported in the attack, which caused damage to several apartments in the building, according to Israeli media reports.


The 50 greatest innovations of 2024

Popular Science

In 1988, we launched the Best of What's New Awards. The original list highlighted "the very things that make our lives more comfortable, more rewarding, more exciting, and more fun," to quote then-Publisher Grant A. Burnett. Now, in 2024, we continue our decades-old tradition of honoring big ideas. We even see hints of our original honorees in this year's list: Sea-Doo and Ford made both lists, 36 years apart. We're proud to bring you promising innovations--from things that make life at home easier to literal out-of-this-world explorations. This is the Best of What's New 2024. Had you asked me at the beginning of 2024 what our best gadgets list would look like, I'd have guessed it would be filled with quirky AI-driven devices like the rabbit R1 or the Humane Ai Pin. "Now with AI" is a phrase that has dominated consumer electronics in the 2020s. These devices promised unadulterated access to the power of neural networks in ways that would seamlessly integrate into our lives without relying on phones or smart fridges. Then, the devices came out. The software is slow and buggy, and the hardware is clunky. Maybe the stand-alone AI device will still have its year, and we'll look back and chuckle at these humble beginnings. In reality, 2024's big breakthrough came from Apple in the form of its long-rumored Vision Pro headset. The device has its own hurdles to clear, but after just a few minutes of using it, it was clear that it's something different, important, and honestly pretty amazing. The list also includes Sony's innovative pro-grade camera, the most accessible drone we've ever used, and a no-fun phone--no fun in a good way, of course. Credible rumors of Apple's VR bounced around the gadget blogs and tech sites for nearly a decade. It was consumer tech's sasquatch in that people claimed to have seen it, but no one knew if it even existed. Then, the Vision Pro emerged from the proverbial forest in February with a surprising design and a massive 3,500 price tag. It also came toting a new R-series chip and a dedicated OS meant for spatial computing.


The Sticky Dilemmas of Pornhub's Next Chapter

WIRED

It was evening in Berlin and Alex Kekesi was surrounded by pornstars. Venus, the international adult entertainment convention, was underway, and Kekesi happened to be at dinner with several well-known creators when the discussion shifted to generative AI. Kekesi listened as a few of the women shared similar stories from set. They expressed frustrations about their likeness being exploited. They talked of having to physically cross out language in their contract before filming.