Goto

Collaborating Authors

 Media


Describing Differences in Image Sets with Natural Language

arXiv.org Artificial Intelligence

How do two sets of images differ? Discerning set-level differences is crucial for understanding model behaviors and analyzing datasets, yet manually sifting through thousands of images is impractical. To aid in this discovery process, we explore the task of automatically describing the differences between two $\textbf{sets}$ of images, which we term Set Difference Captioning. This task takes in image sets $D_A$ and $D_B$, and outputs a description that is more often true on $D_A$ than $D_B$. We outline a two-stage approach that first proposes candidate difference descriptions from image sets and then re-ranks the candidates by checking how well they can differentiate the two sets. We introduce VisDiff, which first captions the images and prompts a language model to propose candidate descriptions, then re-ranks these descriptions using CLIP. To evaluate VisDiff, we collect VisDiffBench, a dataset with 187 paired image sets with ground truth difference descriptions. We apply VisDiff to various domains, such as comparing datasets (e.g., ImageNet vs. ImageNetV2), comparing classification models (e.g., zero-shot CLIP vs. supervised ResNet), summarizing model failure modes (supervised ResNet), characterizing differences between generative models (e.g., StableDiffusionV1 and V2), and discovering what makes images memorable. Using VisDiff, we are able to find interesting and previously unknown differences in datasets and models, demonstrating its utility in revealing nuanced insights.


Rank-without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Models

arXiv.org Artificial Intelligence

Listwise rerankers based on large language models (LLM) are the zero-shot state-of-the-art. However, current works in this direction all depend on the GPT models, making it a single point of failure in scientific reproducibility. Moreover, it raises the concern that the current research findings only hold for GPT models but not LLM in general. In this work, we lift this pre-condition and build for the first time effective listwise rerankers without any form of dependency on GPT. Our passage retrieval experiments show that our best list se reranker surpasses the listwise rerankers based on GPT-3.5 by 13% and achieves 97% effectiveness of the ones built on GPT-4. Our results also show that the existing training datasets, which were expressly constructed for pointwise ranking, are insufficient for building such listwise rerankers. Instead, high-quality listwise ranking data is required and crucial, calling for further work on building human-annotated listwise data resources.


Concept Drift Adaptation in Text Stream Mining Settings: A Comprehensive Review

arXiv.org Artificial Intelligence

Due to the advent and increase in the popularity of the Internet, people have been producing and disseminating textual data in several ways, such as reviews, social media posts, and news articles. As a result, numerous researchers have been working on discovering patterns in textual data, especially because social media posts function as social sensors, indicating peoples' opinions, interests, etc. However, most tasks regarding natural language processing are addressed using traditional machine learning methods and static datasets. This setting can lead to several problems, such as an outdated dataset, which may not correspond to reality, and an outdated model, which has its performance degrading over time. Concept drift is another aspect that emphasizes these issues, which corresponds to data distribution and pattern changes. In a text stream scenario, it is even more challenging due to its characteristics, such as the high speed and data arriving sequentially. In addition, models for this type of scenario must adhere to the constraints mentioned above while learning from the stream by storing texts for a limited time and consuming low memory. In this study, we performed a systematic literature review regarding concept drift adaptation in text stream scenarios. Considering well-defined criteria, we selected 40 papers to unravel aspects such as text drift categories, types of text drift detection, model update mechanism, the addressed stream mining tasks, types of text representations, and text representation update mechanism. In addition, we discussed drift visualization and simulation and listed real-world datasets used in the selected papers. Therefore, this paper comprehensively reviews the concept drift adaptation in text stream mining scenarios.


DRAFT: Dense Retrieval Augmented Few-shot Topic classifier Framework

arXiv.org Artificial Intelligence

With the growing volume of diverse information, the demand for classifying arbitrary topics has become increasingly critical. To address this challenge, we introduce DRAFT, a simple framework designed to train a classifier for few-shot topic classification. DRAFT uses a few examples of a specific topic as queries to construct Customized dataset with a dense retriever model. Multi-query retrieval (MQR) algorithm, which effectively handles multiple queries related to a specific topic, is applied to construct the Customized dataset. Subsequently, we fine-tune a classifier using the Customized dataset to identify the topic. To demonstrate the efficacy of our proposed approach, we conduct evaluations on both widely used classification benchmark datasets and manually constructed datasets with 291 diverse topics, which simulate diverse contents encountered in real-world applications. DRAFT shows competitive or superior performance compared to baselines that use in-context learning, such as GPT-3 175B and InstructGPT 175B, on few-shot topic classification tasks despite having 177 times fewer parameters, demonstrating its effectiveness.


Controllable Music Production with Diffusion Models and Guidance Gradients

arXiv.org Artificial Intelligence

We demonstrate how conditional generation from diffusion models can be used to tackle a variety of realistic tasks in the production of music in 44.1kHz stereo audio with sampling-time guidance. The scenarios we consider include continuation, inpainting and regeneration of musical audio, the creation of smooth transitions between two different music tracks, and the transfer of desired stylistic characteristics to existing audio clips. We achieve this by applying guidance at sampling time in a simple framework that supports both reconstruction and classification losses, or any combination of the two. This approach ensures that generated audio can match its surrounding context, or conform to a class distribution or latent representation specified relative to any suitable pre-trained classifier or embedding model.


A Dating App Tried to Update Its Interface. Unbridled, Horny Chaos Ensued.

Slate

When Aaron* logged on to the kinky, nonmonogamy-focused dating app Feeld on Thursday to finalize plans with a match, the interface wouldn't load. As a middle-aged man in an ethically nonmonogamous relationship, Aaron considers Feeld a great way to meet other like-minded people in his area--and that's exactly what he was hoping to do this past Friday. Someone he had a connection with was in town for one night only, and he wanted to take advantage. He tried logging in again and changing his Wi-Fi connection, but nothing seemed to do the trick. Flummoxed, he took to X, the platform formerly known as Twitter, to see if there was any explanation.


Oxford University Press chooses 'rizz' as the 2023 word of the year

FOX News

Fox News Flash top headlines are here. Check out what's clicking on Foxnews.com. Oxford University Press has named "rizzโ€ณ as its word of the year, highlighting the popularity of a term used by Generation Z to describe someone's ability to attract or seduce another person. It topped "Swiftie" (an enthusiastic fan of Taylor Swift), "situationship" (an informal romantic or sexual relationship) and "prompt" (an instruction given to an artificial intelligence program) in the annual decision by experts at the publisher of the multivolume Oxford English Dictionary. An Oxford English Dictionary is shown at the headquarters of the Associated Press in New York on Aug. 29, 2010. Oxford University Press has named "rizzโ€ณ as its word of the year, highlighting the popularity of a term used by Generation Z to describe someone's ability to attract or seduce another person.


AI's Influence on Music Is Raising Some Difficult Questions

TIME - Tech

Earlier this year, Bad Bunny emphatically rejected rumors that he was about to release a new song with Justin Bieber. "That's fake," he told TIME in an interview for a cover story on his meteoric rise. "You never know what I'm going to do." But last month, a song featuring what sounded like his and Bieber's voices started circulating on TikTok, garnering millions of likes. Bad Bunny hadn't lied in the interview, though: the song was created with AI.


The Wizard of AI โ€“ a film by Alan Warburton

AIHub

One of the highlights of the recent Open Data Institute (ODI) Summit 2023 was the showing of a short film by artist and AI collaborator, Alan Warburton. This video essay was commissioned by the ODI's Data as Culture programme and addresses the cultural impacts of generative AI. The ODI Summit 2023 took place on 7 November and featured keynote presentations, lightening talks, and panel discussions. The event brought together representatives from civil society, academia, industry, and government. Find out more on the ODI website.


American businesses love AI. But what do consumers think?

FOX News

Fox News Flash top headlines are here. Check out what's clicking on Foxnews.com. In early November, Bentley University and Gallup released the results of its 2023 Bentley-Gallup Business and Society Report, which among other topics, focuses a portion of its study on surveying Americans on their opinions of how businesses will use artificial intelligence (AI) technologies in the future. When asked "In general, how much do you trust businesses to use artificial intelligence responsibly?", What is particularly telling, is that across education levels, ethnic background, age groups, and political party, the range of those trusting AI a "lot/some" was only between 17% and 28%.