Personal
Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey
Miyai, Atsuyuki, Yang, Jingkang, Zhang, Jingyang, Ming, Yifei, Lin, Yueqian, Yu, Qing, Irie, Go, Joty, Shafiq, Li, Yixuan, Li, Hai, Liu, Ziwei, Yamasaki, Toshihiko, Aizawa, Kiyoharu
Detecting out-of-distribution (OOD) samples is crucial for ensuring the safety of machine learning systems and has shaped the field of OOD detection. Meanwhile, several other problems are closely related to OOD detection, including anomaly detection (AD), novelty detection (ND), open set recognition (OSR), and outlier detection (OD). To unify these problems, a generalized OOD detection framework was proposed, taxonomically categorizing these five problems. However, Vision Language Models (VLMs) such as CLIP have significantly changed the paradigm and blurred the boundaries between these fields, again confusing researchers. In this survey, we first present a generalized OOD detection v2, encapsulating the evolution of AD, ND, OSR, OOD detection, and OD in the VLM era. Our framework reveals that, with some field inactivity and integration, the demanding challenges have become OOD detection and AD. In addition, we also highlight the significant shift in the definition, problem settings, and benchmarks; we thus feature a comprehensive review of the methodology for OOD detection, including the discussion over other related tasks to clarify their relationship to OOD detection. Finally, we explore the advancements in the emerging Large Vision Language Model (LVLM) era, such as GPT-4V. We conclude this survey with open challenges and future directions.
Generative Learning for Simulation of Vehicle Faults
Kuiper, Patrick, Lin, Sirui, Blanchet, Jose, Tarokh, Vahid
We focus this analysis on the United States' Department of Defense (DoD), where the US Army alone is projected to spend an estimated $5 billion per year (in 2020 dollar terms through 2050), developing and acquiring ground vehicles, where ground vehicles are any vehicles other than aircraft and ships (CBO 2021). Maintaining this enormous investment is critical to ensuring combat readiness across the DoD, where the department spent $90 billion in 2022 on maintaining vehicles across domains: ground, air, and sea (GAO 2022). Predicting requirements is critical to an effective maintenance program. The application of statistics towards vehicle maintenance prediction is often referred to as predictive maintenance. Recognizing the importance of predictive maintenance, in the 2022 National Defense Authorization Act (NDAA) Congress required the DoD Inspector General Office to review predictive maintenance practices, originally established by DoD directives in 2002 and 2007 (DoDIG 2023).
Engaging with Children's Artwork in Mixed Visual-Ability Families
Chheda-Kothary, Arnavi, Wobbrock, Jacob O., Froehlich, Jon E.
We present two studies exploring how blind or low-vision (BLV) family members engage with their sighted children's artwork, strategies to support understanding and interpretation, and the potential role of technology, such as AI, therein. Our first study involved 14 BLV individuals, and the second included five groups of BLV individuals with their children. Through semi-structured interviews with AI descriptions of children's artwork and multi-sensory design probes, we found that BLV family members value artwork engagement as a bonding opportunity, preferring the child's storytelling and interpretation over other nonvisual representations. Additionally, despite some inaccuracies, BLV family members felt that AI-generated descriptions could facilitate dialogue with their children and aid self-guided art discovery. We close with specific design considerations for supporting artwork engagement in mixed visual-ability families, including enabling artwork access through various methods, supporting children's corrections of AI output, and distinctions in context vs. content and interpretation vs. description of children's artwork.
Canadian Olympic Committee says spying scandal 'could tarnish' women's Tokyo gold medal
The drone scandal surrounding the Canadian women's soccer team could have bigger implications than just this year's Games in Paris. Head coach Bev Priestman was removed from her position on Thursday night after two staff members were sent home from Paris after an investigation found that analyst Joseph Lombardi had used a drone to spy on New Zealand's practice sessions. Head coach Beverly Priestman reacts during the Women's Gold Medal match between Canada and Sweden on day 14 of the Tokyo 2020 Olympic Games at International Stadium Yokohama on Aug. 6, 2021 in Yokohama, Kanagawa, Japan. "Over the past 24 hours, additional information has come to our attention regarding previous drone use against opponents, predating the Paris 2024 Olympic Games," Canada Soccer CEO Kevin Blue said in a statement. "In light of these new revelations, Canada Soccer has made the decision to suspend Women's National Soccer Team Head Coach, Bev Priestman for the remainder of the Paris 2024 Olympic Games, and until the completion of our recently announced independent external review."
Congratulations to the #ICML2024 award winners
VideoPoet employs a decoder-only transformer architecture that processes multimodal inputs – including images, videos, text, and audio. The training protocol follows that of Large Language Models (LLMs), consisting of two stages: pretraining and task-specific adaptation. During pretraining, VideoPoet incorporates a mixture of multimodal generative objectives within an autoregressive Transformer framework. The pretrained LLM serves as a foundation that can be adapted for a range of video generation tasks. We present empirical results demonstrating the model's state-of-the-art capabilities in zero-shot video generation, specifically highlighting the ability to generate high-fidelity motions.
Self-Directed Synthetic Dialogues and Revisions Technical Report
Lambert, Nathan, Schoelkopf, Hailey, Gokaslan, Aaron, Soldaini, Luca, Pyatkin, Valentina, Castricato, Louis
Synthetic data has become an important tool in the fine-tuning of language models to follow instructions and solve complex problems. Nevertheless, the majority of open data to date is often lacking multi-turn data and collected on closed models, limiting progress on advancing open fine-tuning methods. We introduce Self Directed Synthetic Dialogues (SDSD), an experimental dataset consisting of guided conversations of language models talking to themselves. The dataset consists of multi-turn conversations generated with DBRX, Llama 2 70B, and Mistral Large, all instructed to follow a conversation plan generated prior to the conversation. We also explore including principles from Constitutional AI and other related works to create synthetic preference data via revisions to the final conversation turn. We hope this work encourages further exploration in multi-turn data and the use of open models for expanding the impact of synthetic data.
Why Colin Kaepernick Is Starting an AI Company
When NFL quarterback Colin Kaepernick began kneeling during the national anthem to protest police brutality and racial injustice in 2016, he soon found himself out of a job, eventually moving onto other ventures in media and entertainment. Today, he's entering the AI industry by launching a project he says he hopes will allow others to bypass "gatekeeping:" an artificial intelligence platform called Lumi. The new subscription-based platform aims to provide tools for storytellers to create, illustrate, publish and monetize their ideas. The company has raised 4 million in funding led by Alexis Ohanian's Seven Seven Six, and its product went live today, July 24. In an interview with TIME, Kaepernick says this project can be viewed as an extension of his activism.
AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Game
Chi, Yizhou, Mao, Lingjun, Tang, Zineng
Strategic social deduction games serve as valuable testbeds for evaluating the understanding and inference skills of language models, offering crucial insights into social science, artificial intelligence, and strategic gaming. This paper focuses on creating proxies of human behavior in simulated environments, with Among Us utilized as a tool for studying simulated human behavior. The study introduces a text-based game environment, named AmongAgents, that mirrors the dynamics of Among Us. Players act as crew members aboard a spaceship, tasked with identifying impostors who are sabotaging the ship and eliminating the crew. Within this environment, the behavior of simulated language agents is analyzed. The experiments involve diverse game sequences featuring different configurations of Crewmates and Impostor personality archetypes. Our work demonstrates that state-of-the-art large language models (LLMs) can effectively grasp the game rules and make decisions based on the current context. This work aims to promote further exploration of LLMs in goal-oriented games with incomplete information and complex action spaces, as these settings offer valuable opportunities to assess language model performance in socially driven scenarios.
Carvalho, who unplugged school AI chatbot, wants task force to tell him what went wrong
Alberto Carvalho, who remains determined to bring artificial intelligence into district classrooms despite the collapse of the technology company leading the effort, will appoint a task force to examine what went wrong and how to move forward. The schools chief announced the task force in an interview with The Times in advance of Tuesday's annual address to administrators, which is akin to a state-of-the-schools speech. In his public address, Carvalho is expected to highlight academic progress and L.A. Unified School District initiatives. In a recent appearance, he said he was hopeful that standardized test scores would rise at all grade levels in math and English. Although school districts throughout the state have received results -- and can make them public if they wish -- the state has not yet released local or statewide scores.
The Contribution of XAI for the Safe Development and Certification of AI: An Expert-Based Analysis
Fresz, Benjamin, Göbels, Vincent Philipp, Omri, Safa, Brajovic, Danilo, Aichele, Andreas, Kutz, Janika, Neuhüttler, Jens, Huber, Marco F.
Developing and certifying safe - or so-called trustworthy - AI has become an increasingly salient issue, especially in light of upcoming regulation such as the EU AI Act. In this context, the black-box nature of machine learning models limits the use of conventional avenues of approach towards certifying complex technical systems. As a potential solution, methods to give insights into this black-box - devised in the field of eXplainable AI (XAI) - could be used. In this study, the potential and shortcomings of such methods for the purpose of safe AI development and certification are discussed in 15 qualitative interviews with experts out of the areas of (X)AI and certification. We find that XAI methods can be a helpful asset for safe AI development, as they can show biases and failures of ML-models, but since certification relies on comprehensive and correct information about technical systems, their impact is expected to be limited.