Media
Joyful: Joint Modality Fusion and Graph Contrastive Learning for Multimodal Emotion Recognition
Li, Dongyuan, Wang, Yusong, Funakoshi, Kotaro, Okumura, Manabu
Multimodal emotion recognition aims to recognize emotions for each utterance of multiple modalities, which has received increasing attention for its application in human-machine interaction. Current graph-based methods fail to simultaneously depict global contextual features and local diverse uni-modal features in a dialogue. Furthermore, with the number of graph layers increasing, they easily fall into over-smoothing. In this paper, we propose a method for joint modality fusion and graph contrastive learning for multimodal emotion recognition (Joyful), where multimodality fusion, contrastive learning, and emotion recognition are jointly optimized. Specifically, we first design a new multimodal fusion mechanism that can provide deep interaction and fusion between the global contextual and uni-modal specific features. Then, we introduce a graph contrastive learning framework with inter-view and intra-view contrastive losses to learn more distinguishable representations for samples with different sentiments. Extensive experiments on three benchmark datasets indicate that Joyful achieved state-of-the-art (SOTA) performance compared to all baselines.
Outage Performance and Novel Loss Function for an ML-Assisted Resource Allocation: An Exact Analytical Framework
Simmons, Nidhi, Simmons, David E, Yacoub, Michel Daoud
We introduce a novel loss function to minimize the outage probability of an ML-based resource allocation system. A single-user multi-resource greedy allocation strategy constitutes our application scenario, for which an ML binary classification predictor assists in selecting a resource satisfying the established outage criterium. While other resource allocation policies may be suitable, they are not the focus of our study. Instead, our primary emphasis is on theoretically developing this loss function and leveraging it to train an ML model to address the outage probability challenge. With no access to future channel state information, this predictor foresees each resource's likely future outage status. When the predictor encounters a resource it believes will be satisfactory, it allocates it to the user. Our main result establishes exact and asymptotic expressions for this system's outage probability. These expressions reveal that focusing solely on the optimization of the per-resource outage probability conditioned on the ML predictor recommending resource allocation (a strategy that appears to be most appropriate) may produce inadequate predictors that reject every resource. They also reveal that focusing on standard metrics, like precision, false-positive rate, or recall, may not produce optimal predictors. With our result, we formulate a theoretically optimal, differentiable loss function to train our predictor. We then compare predictors trained using this and traditional loss functions namely, binary cross-entropy (BCE), mean squared error (MSE), and mean absolute error (MAE). In all scenarios, predictors trained using our novel loss function provide superior outage probability performance. Moreover, in some cases, our loss function outperforms predictors trained with BCE, MAE, and MSE by multiple orders of magnitude.
5 things to do first if you got a new Mac
CyberGuy explains how Walmart is using artificial intelligence to enhance the shopping experience. You know that feeling when you unbox a new Mac for the first time? You can't help but admire how sleek and smooth it looks and how bright and beautiful it glows when you turn it on. And don't get me started on those crisp and clean keys that make typing a breeze. Before we jump into all the cool stuff you can do with your Mac, there are some important things you need to set up first.
Google DeepMind's AI Pop Star Clone Will Freak You Out
Even if you didn't watch last weekend's episode of Saturday Night Live, you still probably saw it. You may already even know what "it" I'm talking about: Timothée Chalamet, and other similarly-dressed cast members, booty-shaking in tiny little red undies. He was, the sketch goes, "an Australian YouTube twink turned indie pop star and model turned HBO actor Troye Sivan being played by an American actor who can't do an Australian accent." Chalamet and his cohort were Troye Sivan Sleep Demons, and they'd been haunting straight women all over the place. It was a funny bit and, ironically, the least nightmarish Sivan impression to come out this week.
YouTube launches AI tool that lets you CLONE pop stars' voices - so, would this Charlie Puth track fool you?
YouTube has sidestepped controversies surrounding the use of artificial intelligence (AI) to generate new music with a new tool that clones singers' voices. The new feature, called Dream Track, is available in YouTube Shorts – the Google-owned platform's answer to TikTok that lets users post short videos. Users can enter a prompt about what sort of music style they want – such as'upbeat' or'ballad' – and select the artist they want the AI to imitate. Nine artists have allowed their voice to be copied for the tool, including Alec Benjamin, Charlie Puth, Charli XCX, John Legend, Sia and Troye Sivan. YouTube posted a short clip of what the clone version of US singer Charlie Puth sounds like – and it's impressively close to the real thing.
Sheryl Crow admits she's 'terrified' by AI, fears of technology inspired new song
At her Rock & Roll Hall of Fame induction interview backstage, Sheryl Crow told reporters that AI inspired her to write a song to deal with her fear of the technology. Sheryl Crow found inspiration for her new album from artificial intelligence, though she said the technology left her "terrified." At her induction to the Rock & Roll Hall of Fame earlier this month, Crow said she hadn't intended to do another album, planning instead to just release songs. But then "when the whole AI thing started coming out, particularly with the Beatles thing, and also having witnessed how AI is being used in my art form, I wrote a song about it." She continued, "I was terrified, and where do I go when I'm terrified? I go to my studio," adding, "And I found myself writing just one thing after another, and lo and behold I had 10 songs."
HungerGist: An Interpretable Predictive Model for Food Insecurity
Ahn, Yongsu, Yan, Muheng, Lin, Yu-Ru, Wang, Zian
The escalating food insecurity in Africa, caused by factors such as war, climate change, and poverty, demonstrates the critical need for advanced early warning systems. Traditional methodologies, relying on expert-curated data encompassing climate, geography, and social disturbances, often fall short due to data limitations, hindering comprehensive analysis and potential discovery of new predictive factors. To address this, this paper introduces "HungerGist", a multi-task deep learning model utilizing news texts and NLP techniques. Using a corpus of over 53,000 news articles from nine African countries over four years, we demonstrate that our model, trained solely on news data, outperforms the baseline method trained on both traditional risk factors and human-curated keywords. In addition, our method has the ability to detect critical texts that contain interpretable signals known as "gists." Moreover, our examination of these gists indicates that this approach has the potential to reveal latent factors that would otherwise remain concealed in unstructured texts.
Emotion-Aware Music Recommendation System: Enhancing User Experience Through Real-Time Emotional Context
Babu, Tina, Nair, Rekha R, A, Geetha
This study addresses the deficiency in conventional music recommendation systems by focusing on the vital role of emotions in shaping users music choices. These systems often disregard the emotional context, relying predominantly on past listening behavior and failing to consider the dynamic and evolving nature of users emotional preferences. This gap leads to several limitations. Users may receive recommendations that do not match their current mood, which diminishes the quality of their music experience. Furthermore, without accounting for emotions, the systems might overlook undiscovered or lesser-known songs that have a profound emotional impact on users. To combat these limitations, this research introduces an AI model that incorporates emotional context into the song recommendation process. By accurately detecting users real-time emotions, the model can generate personalized song recommendations that align with the users emotional state. This approach aims to enhance the user experience by offering music that resonates with their current mood, elicits the desired emotions, and creates a more immersive and meaningful listening experience. By considering emotional context in the song recommendation process, the proposed model offers an opportunity for a more personalized and emotionally resonant musical journey.
Countering Misinformation via Emotional Response Generation
Russo, Daniel, Kaszefski-Yaschuk, Shane Peter, Staiano, Jacopo, Guerini, Marco
The proliferation of misinformation on social media platforms (SMPs) poses a significant danger to public health, social cohesion and ultimately democracy. Previous research has shown how social correction can be an effective way to curb misinformation, by engaging directly in a constructive dialogue with users who spread -- often in good faith -- misleading messages. Although professional fact-checkers are crucial to debunking viral claims, they usually do not engage in conversations on social media. Thereby, significant effort has been made to automate the use of fact-checker material in social correction; however, no previous work has tried to integrate it with the style and pragmatics that are commonly employed in social media communication. To fill this gap, we present VerMouth, the first large-scale dataset comprising roughly 12 thousand claim-response pairs (linked to debunking articles), accounting for both SMP-style and basic emotions, two factors which have a significant role in misinformation credibility and spreading. To collect this dataset we used a technique based on an author-reviewer pipeline, which efficiently combines LLMs and human annotators to obtain high-quality data. We also provide comprehensive experiments showing how models trained on our proposed dataset have significant improvements in terms of output quality and generalization capabilities.