Media
The NeurIPS 2023 Machine Learning for Audio Workshop: Affective Audio Benchmarks and Novel Data
Baird, Alice, Manzelli, Rachel, Tzirakis, Panagiotis, Gagne, Chris, Li, Haoqi, Allen, Sadie, Dieleman, Sander, Kulis, Brian, Narayanan, Shrikanth S., Cowen, Alan
The NeurIPS 2023 Machine Learning for Audio Workshop brings together machine learning (ML) experts from various audio domains. There are several valuable audio-driven ML tasks, from speech emotion recognition to audio event detection, but the community is sparse compared to other ML areas, e.g., computer vision or natural language processing. A major limitation with audio is the available data; with audio being a time-dependent modality, high-quality data collection is time-consuming and costly, making it challenging for academic groups to apply their often state-of-the-art strategies to a larger, more generalizable dataset. In this short white paper, to encourage researchers with limited access to large-datasets, the organizers first outline several open-source datasets that are available to the community, and for the duration of the workshop are making several propriety datasets available. Namely, three vocal datasets, Hume-Prosody, Hume-VocalBurst, an acted emotional speech dataset Modulate-Sonata, and an in-game streamer dataset Modulate-Stream. We outline the current baselines on these datasets but encourage researchers from across audio to utilize them outside of the initial baseline tasks.
Ask a doc: 'Do I need to wash my face every night?'
Fox News Flash top headlines are here. Check out what's clicking on Foxnews.com. You likely brush your teeth every night -- but you may not realize how important it is to wash your face before going to bed every night, too. To learn more about why face-washing should be on your bedtime to-do list, Fox News Digital asked two dermatologists for the clean truth. When you look in the mirror, you likely don't see the buildup of substances on your face from that day.
The Kate Middleton Situation Was Already a Mess. The Royals Have Now Made It a Permanent Crisis.
It's been just over a week since Kate Middleton, the internet's favorite "missing person," claimed that a photoshopped image of her with her children on U.K. Mother's Day was edited by her, for unspecified reasons. Then, on Monday, we had our first recorded sighting of the princess, out shopping with Prince William at the Royal Farms Windsor Farm Shop, near Windsor Castle. The video was released by TMZ and the Sun, and stills from it were plastered on the front pages of all the British tabloids Tuesday. Supposedly, it was taken by a 40-year-man, Nelson Silva, who lives nearby and was quoted in TMZ as saying: "Kate looked happy and relaxed. They look happy just to be able to go to a shop and mingle.
'I turned C-3PO into a lightsaber-wielding psychopath': a week with the Star Wars Unlimited card game
One of the most appealing aspects of games set in the Star Wars universe is that you get to concoct scenes and stories we would never see in the movies. Whether you're playing Knights of the Old Republic, Jedi: Fallen Order or the old Star Wars role-playing board game designed by Greg Costikyan in the 1990s, there will be individual moments unrepeatable on the big screen. I know this, because I just won a round of the new trading card game Star Wars Unlimited thanks to a heroic C-3PO wielding Luke Skywalker's lightsaber. On a basic level, Star Wars Unlimited works like most modern trading card games, such as Yu-Gi-Oh! You and an opponent each have a deck of cards, most of which feature a single character or vehicle, with a number for health and another number for power/damage.
Kids' Cartoons Get a Free Pass From YouTube's Deepfake Disclosure Rules
YouTube has updated its rulebook for the era of deepfakes. Starting today, anyone uploading video to the platform must disclose certain uses of synthetic media, including generative AI, so viewers know what they're seeing isn't real. YouTube says it applies to "realistic" altered media such as "making it appear as if a real building caught fire" or swapping "the face of one individual with another's." The new policy shows YouTube taking steps that could help curb the spread of AI-generated misinformation as the US presidential election approaches. It is also striking for what it permits: AI-generated animations aimed at kids are not subject to the new synthetic content disclosure rules.
Algorithmic Collective Action in Recommender Systems: Promoting Songs by Reordering Playlists
Baumann, Joachim, Mendler-Dรผnner, Celestine
We investigate algorithmic collective action in transformer-based recommender systems. Our use case is a collective of fans aiming to promote the visibility of an artist by strategically placing one of their songs in the existing playlists they control. The success of the collective is measured by the increase in test-time recommendations of the targeted song. We introduce two easily implementable strategies towards this goal and test their efficacy on a publicly available recommender system model released by a major music streaming platform. Our findings reveal that even small collectives (controlling less than 0.01% of the training data) can achieve up 25x amplification of recommendations by strategically choosing the position at which to insert the song. We then focus on investigating the externalities of the strategy. We find that the performance loss for the platform is negligible, and the recommendations of other songs are largely preserved, minimally impairing the user experience of participants. Moreover, the costs are evenly distributed among other artists. Taken together, our findings demonstrate how collective action strategies can be effective while not necessarily being adversarial, raising new questions around incentives, social dynamics, and equilibria in recommender systems.
Can AI Outperform Human Experts in Creating Social Media Creatives?
Park, Eunkyung, Wong, Raymond K., Kwon, Junbum
Artificial Intelligence has outperformed human experts in functional tasks such as chess and baduk. How about creative tasks? This paper evaluates AI's capability in the creative domain compared to human experts, which little research has been conducted so far. We propose a novel Prompt-for-Prompt to generate social media creatives via prompt augmentation by Large Language Models. We take the most popular Instagram posts (with the biggest number of like clicks) in top brands' Instagram accounts to create social media creatives. We give GPT 4 several prompt instructions with text descriptions to generate the most effective prompts for cutting-edge text-to-image generators: Midjourney, DALL E 3, and Stable Diffusion. LLM-augmented prompts can boost AI's abilities by adding objectives, engagement strategy, lighting and brand consistency for social media image creation. We conduct an extensive human evaluation experiment, and find that AI excels human experts, and Midjourney is better than the other text-to-image generators. Surprisingly, unlike conventional wisdom in the social media industry, prompt instruction including eye-catching shows much poorer performance than those including natural. Regarding the type of creatives, AI improves creatives with animals or products but less with real people. Also, AI improves creatives with short text descriptions more than with long text descriptions, because there is more room for AI to augment prompts with shorter descriptions.
The Use of Generative Search Engines for Knowledge Work and Complex Tasks
Suri, Siddharth, Counts, Scott, Wang, Leijie, Chen, Chacha, Wan, Mengting, Safavi, Tara, Neville, Jennifer, Shah, Chirag, White, Ryen W., Andersen, Reid, Buscher, Georg, Manivannan, Sathish, Rangan, Nagu, Yang, Longqi
Until recently, search engines were the predominant method for people to access online information. The recent emergence of large language models (LLMs) has given machines new capabilities such as the ability to generate new digital artifacts like text, images, code etc., resulting in a new tool, a generative search engine, which combines the capabilities of LLMs with a traditional search engine. Through the empirical analysis of Bing Copilot (Bing Chat), one of the first publicly available generative search engines, we analyze the types and complexity of tasks that people use Bing Copilot for compared to Bing Search. Findings indicate that people use the generative search engine for more knowledge work tasks that are higher in cognitive complexity than were commonly done with a traditional search engine.
Dated Data: Tracing Knowledge Cutoffs in Large Language Models
Cheng, Jeffrey, Marone, Marc, Weller, Orion, Lawrie, Dawn, Khashabi, Daniel, Van Durme, Benjamin
Released Large Language Models (LLMs) are often paired with a claimed knowledge cutoff date, or the dates at which training data was gathered. Such information is crucial for applications where the LLM must provide up to date information. However, this statement only scratches the surface: do all resources in the training data share the same knowledge cutoff date? Does the model's demonstrated knowledge for these subsets closely align to their cutoff dates? In this work, we define the notion of an effective cutoff. This is distinct from the LLM designer reported cutoff and applies separately to sub-resources and topics. We propose a simple approach to estimate effective cutoffs on the resource-level temporal alignment of an LLM by probing across versions of the data. Using this analysis, we find that effective cutoffs often differ from reported cutoffs. To understand the root cause of this observation, we conduct a direct large-scale analysis on open pre-training datasets. Our analysis reveals two reasons for these inconsistencies: (1) temporal biases of CommonCrawl data due to non-trivial amounts of old data in new dumps and (2) complications in LLM deduplication schemes involving semantic duplicates and lexical near-duplicates. Overall, our results show that knowledge cutoffs are not as simple as they have seemed and that care must be taken both by LLM dataset curators as well as practitioners who seek to use information from these models.
Community Needs and Assets: A Computational Analysis of Community Conversations
Chowdhury, Md Towhidul Absar, Sharma, Naveen, KhudaBukhsh, Ashiqur R.
A community needs assessment is a tool used by non-profits and government agencies to quantify the strengths and issues of a community, allowing them to allocate their resources better. Such approaches are transitioning towards leveraging social media conversations to analyze the needs of communities and the assets already present within them. However, manual analysis of exponentially increasing social media conversations is challenging. There is a gap in the present literature in computationally analyzing how community members discuss the strengths and needs of the community. To address this gap, we introduce the task of identifying, extracting, and categorizing community needs and assets from conversational data using sophisticated natural language processing methods. To facilitate this task, we introduce the first dataset about community needs and assets consisting of 3,511 conversations from Reddit, annotated using crowdsourced workers. Using this dataset, we evaluate an utterance-level classification model compared to sentiment classification and a popular large language model (in a zero-shot setting), where we find that our model outperforms both baselines at an F1 score of 94% compared to 49% and 61% respectively. Furthermore, we observe through our study that conversations about needs have negative sentiments and emotions, while conversations about assets focus on location and entities. The dataset is available at https://github.com/towhidabsar/CommunityNeeds.