Goto

Collaborating Authors

 kirby


VideoGameBench: Can Vision-Language Models complete popular video games?

Zhang, Alex L., Griffiths, Thomas L., Narasimhan, Karthik R., Press, Ofir

arXiv.org Artificial Intelligence

Vision-language models (VLMs) have achieved strong results on coding and math benchmarks that are challenging for humans, yet their ability to perform tasks that come naturally to humans--such as perception, spatial navigation, and memory management--remains understudied. Real video games are crafted to be intuitive for humans to learn and master by leveraging innate inductive biases, making them an ideal testbed for evaluating such capabilities in VLMs. To this end, we introduce VideoGameBench, a benchmark consisting of 10 popular video games from the 1990s that VLMs directly interact with in real-time. VideoGameBench challenges models to complete entire games with access to only raw visual inputs and a high-level description of objectives and controls, a significant departure from existing setups that rely on game-specific scaffolding and auxiliary information. We keep three of the games secret to encourage solutions that generalize to unseen environments. Our experiments show that frontier vision-language models struggle to progress beyond the beginning of each game. We find inference latency to be a major limitation of frontier models in the real-time setting; therefore, we introduce VideoGameBench Lite, a setting where the game pauses while waiting for the LM's next action. The best performing model, Gemini 2.5 Pro, completes only 0.48% of VideoGameBench and 1.6% of VideoGameBench Lite. We hope that the formalization of the human skills mentioned above into this benchmark motivates progress in these research directions.


The Great Language Flattening

The Atlantic - Technology

In at least one crucial way, AI has already won its campaign for global dominance. An unbelievable volume of synthetic prose is published every moment of every day--heaping piles of machine-written news articles, text messages, emails, search results, customer-service chats, even scientific research. Chatbots learned from human writing. Now the influence may run in the other direction. Some people have hypothesized that the proliferation of generative-AI tools such as ChatGPT will seep into human communication, that the terse language we use when prompting a chatbot may lead us to dispose of any niceties or writerly flourishes when corresponding with friends and colleagues.


Drone ban in New Jersey sees restrictions in 22 towns due to 'special security reasons'

Daily Mail - Science & tech

The Federal Aviation Administration (FAA) has announced a temporary drone ban in New Jersey, citing'special security reasons.' At least 22 towns in central and northern New Jersey fall under the alert, which is in place until at least January 17. The temporary flight restriction (TFR) areas include parts of Camden, Gloucester City, Winslow Township, Evesham, Hancock's Bridge in Lower Alloways Township in Salem County, Westampton, Burlington, and Hamilton in Mercer County. Flying drones is also banned in Bridgewater, Cedar Grove, North Brunswick, Metuchen, South Brunswick, Edison, Branchburg, Sewaren, Jersey City, Harrison, Elizabeth, Bayonne, Clifton, and Kearny. The FAA warned that'deadly force' could be used against drones that present an'imminent security threat.'


LIZ PEEK: Biden's drone stonewalling finally fulfills president's campaign promise

FOX News

Joe Biden has finally delivered on one of his central 2020 campaign promises: he has united the country. Everyone – Republicans, Democrats, mayors, governors, average Joes – everyone is furious that the Biden White House is stonewalling us about the many drones swarming over New Jersey, New York and several other states. It is the perfect coda to the Biden presidency: a White House that is dishonest, scared and inept. John Kirby, White House spokesperson, has blithely parroted nonsense about people confusing what are quite evidently highly sophisticated surveillance machines with "manned aircraft" and "inaccurate sightings." "We have not been able to, and neither have state or local law enforcement authorities, corroborate any of the reported visual sightings" said Kirby in a recent briefing.


Drones flying over the country are 'lawful,' nothing indicates a 'public safety risk,' says top WH official

FOX News

White House National Security Communications Adviser John Kirby said the White House's assessment, in coordination with the FBI and state and local officials, is that the mysterious drones flying over the country are in fact "legal" and "lawful." Kirby told Fox News anchor Bret Baier Monday on "Special Report" they've examined roughly 5,000 sightings and to date, their analysis is "lawful, legal, commercial hobbyist and even law enforcement aircraft activity," is responsible for the sightings. "Some of it's manned, some of it's unmanned. We absolutely acknowledge that a lot of these are probably drones, but they're flying legally. And it is legal to fly drones in non-restricted airspace as long as you're registered with the [Federal Aviation Administration] FAA and there's thousands and thousands of these kinds of flights every single day," he added.


Pentagon should assume drone 'invasion' is 'not friendly,' New Jersey lawmaker says: 'Wake up!'

FOX News

Rep. Chris Smith, R-N.J., gives an update on efforts to find answers regarding the mysterious drone sightings across the Northeast on'Your World.' For nearly a month, Americans in the northeast have spotted what appear to be drones flying through U.S. airspace with no definitive conclusions from the federal government. The White House has responded with some information, but one New Jersey lawmaker is demanding the federal government "wake up" and get to the bottom of the "very threatening situation." "We need to wake up!" Rep. Chris Smith, R-N.J., said Friday on "Your World." "To go about a month of doing nothing, and again, this is why I'm so upset with the Pentagon. To just say again, nothing to see here, puts our people at risk." "We need to identify, identify, identify and then take appropriate action. We can't allow this invasion of drones that we have no idea what their intent is other than -- the presumption should be they're not friendly. If they turn out to be friendly, break out the champagne bottle. More than three weeks after dozens of purported drones began appearing in the New Jersey night sky, the public has still been offered no clear insight on what the phenomenon actually is. The drones were first reported around Nov. 18, and have been spotted every night since, flying from dusk to about 11 p.m. Reports have ranged from four to 180 sightings per night, according to New Jersey Assemblywoman Dawn Fantasia, who relayed a briefing given by law enforcement. Residents in states like New Jersey, Connecticut and New York, as well as state and federal lawmakers, are demanding answers. White House National Security Communications Advisor John Kirby said Thursday that a number of the purported drone sightings spotted over New Jersey for the past several weeks appear to be piloted aircraft. "We have no evidence at this time that the reported drone sightings pose a national security or a public safety threat, or have a foreign nexus," Kirby told reporters at the White House press briefing. "The Department of Homeland Security and the FBI are investigating these sightings, and they're working closely with state and local law enforcement to provide resources using numerous detection methods to better understand their origin." "Using very sophisticated electronic detection technologies provided by federal authorities, we have not been able to, and neither have state or local law enforcement authorities, corroborate any of the reported visual sightings," he said. "To the contrary, upon review of available imagery, it appears that many of the reported sightings are actually manned aircraft that are being operated lawfully.


John Kirby grilled on mysterious New Jersey drone sightings: 'Why don't we know?'

FOX News

White House National Security Communications Advisor John Kirby responds to more questions over the aerial systems on'The Story.' White House National Security Communications Advisor John Kirby maintained that the government still lacks definitive answers regarding the nature of reported drone sightings as public frustration intensifies. "Many of the corroborated sightings have turned out to be piloted aircraft. I didn't say all of them, and what I said was those are the ones we were able to corroborate," Kirby said on "The Story." "There certainly is ones that we have not been able to, and we don't know the answer to it, and I strongly recommend that for folks that are seeing these things and documenting them to share that as they can with the Department of Homeland Security and the FBI." In a Wednesday letter to Biden, New Jersey Gov. Phil Murphy asked the president for more federal resources to address drone sightings, noting that the federal law limits the ability of state and local law enforcement to counter drones.


White House accused of flying drone cover up as New Jersey residents vow to shoot them down - live updates

Daily Mail - Science & tech

Reports of mysterious drone sightings in New Jersey have now spread to multiple states, as residents and local officials demand answers from the US Government. Numerous'car-sized' drones have been seen hovering throughout the state since mid-November, sometimes appearing in groups and often remaining in the same place for hours at a time. The first drone sightings appeared over the US Army's Picatinny Arsenal and over President-elect Donald Trump's golf course in Bedminster on November 18. But reports of varying levels of credibility have now spread to at least 12 counties throughout the Garden State, as well as eastern Pennsylvania and Orange County, New York. The FBI and other agencies are investigating, but the Department of Homeland Security said Wednesday: 'We have no more information as to where these drones are coming from, where they're launching from, where they're landing.'


Many New Jersey 'drone' sightings are lawfully operated manned aircraft, White House says

FOX News

White House National Security spokesman John Kirby addressed the sightings of "drones" over New Jersey's skies, denying that any evidence suggests a foreign adversary is responsible. White House National Security Communications Adviser John Kirby claimed Thursday that many of the purported drone sightings spotted over New Jersey for the past several weeks are actually lawfully operated manned aircraft. "We have no evidence at this time that the reported drone sightings pose a national security or a public safety threat, or have a foreign nexus," Kirby told reporters at the daily White House press briefing. "The Department of Homeland Security (DHS) and the FBI are investigating these sightings, and they're working closely with state and local law enforcement to provide resources using numerous detection methods to better understand their origin." "Using very sophisticated electronic detection technologies provided by federal authorities, we have not been able to, and neither have state or local law enforcement authorities, corroborate any of the reported visual sightings," Kirby said.


Finding structure in logographic writing with library learning

Jiang, Guangyuan, Hofer, Matthias, Mao, Jiayuan, Wong, Lionel, Tenenbaum, Joshua B., Levy, Roger P.

arXiv.org Artificial Intelligence

One hallmark of human language is its combinatoriality -- reusing a relatively small inventory of building blocks to create a far larger inventory of increasingly complex structures. In this paper, we explore the idea that combinatoriality in language reflects a human inductive bias toward representational efficiency in symbol systems. We develop a computational framework for discovering structure in a writing system. Built on top of state-of-the-art library learning and program synthesis techniques, our computational framework discovers known linguistic structures in the Chinese writing system and reveals how the system evolves towards simplification under pressures for representational efficiency. We demonstrate how a library learning approach, utilizing learned abstractions and compression, may help reveal the fundamental computational principles that underlie the creation of combinatorial structures in human cognition, and offer broader insights into the evolution of efficient communication systems.