This paper describes sound event localization and detection (SELD) for spatial audio recordings captured by firstorder ambisonics (FOA) microphones. In this task, one may train a deep neural network (DNN) using FOA data annotated with the classes and directions of arrival (DOAs) of sound events. However, the performance of this approach is severely bounded by the amount of annotated data. To overcome this limitation, we propose a novel method of pretraining the feature extraction part of the DNN in a self-supervised manner. We use spatial audio-visual recordings abundantly available as virtual reality contents. Assuming that sound objects are concurrently observed by the FOA microphones and the omni-directional camera, we jointly train audio and visual encoders with contrastive learning such that the audio and visual embeddings of the same recording and DOA are made close. A key feature of our method is that the DOA-wise audio embeddings are jointly extracted from the raw audio data, while the DOA-wise visual embeddings are separately extracted from the local visual crops centered on the corresponding DOA. This encourages the latent features of the audio encoder to represent both the classes and DOAs of sound events. The experiment using the DCASE2022 Task 3 dataset of 20 hours shows non-annotated audio-visual recordings of 100 hours reduced the error score of SELD from 36.4 pts to 34.9 pts.

Instruction-following is crucial for building AI agents with large language models (LLMs), as these models must adhere strictly to user-provided constraints and guidelines. However, LLMs often fail to follow even simple and clear instructions. To improve instruction-following behavior and prevent undesirable outputs, a deeper understanding of how LLMs' internal states relate to these outcomes is required. Our analysis of LLM internal states reveal a dimension in the input embedding space linked to successful instruction-following. We demonstrate that modifying representations along this dimension improves instruction-following success rates compared to random changes, without compromising response quality. Further investigation reveals that this dimension is more closely related to the phrasing of prompts rather than the inherent difficulty of the task or instructions. This discovery also suggests explanations for why LLMs sometimes fail to follow clear instructions and why prompt engineering is often effective, even when the content remains largely unchanged. This work provides insight into the internal workings of LLMs' instruction-following, paving the way for reliable LLM agents. Given the potential of large language models (LLMs), there has been significant interest in utilizing these models to build personal AI agents. For instance, one could imagine deploying an LLM as a personal healthcare assistant, such as a fitness or nutrition planner, or for psychological counseling (Li et al., 2024b; Wang et al., 2023; Tu et al., 2024). Compared to traditional machine learningbased AI agents, LLMs offer the advantage of being easily adaptable through prompting, allowing users to provide guidelines and personal information without the need to retrain model weights. Instruction-following is critical in the development of personal AI agents with LLMs through prompts because these models must adhere to the constraints and guidelines to ensure safe and trustworthy interactions. For example, suppose an LLM is building a personal fitness plan for a user with knee problems.

The scarcity of high-quality and multi-task singing datasets significantly hinders the development of diverse controllable and personalized singing tasks, as existing singing datasets suffer from low quality, limited diversity of languages and singers, absence of multi-technique information and realistic music scores, and poor task suitability. To tackle these problems, we present GTSinger, a large Global, multi-Technique, free-to-use, high-quality singing corpus with realistic music scores, designed for all singing tasks, along with its benchmarks. Particularly, (1) we collect 80.59 hours of high-quality singing voices, forming the largest recorded singing dataset; (2) 20 professional singers across nine widely spoken languages offer diverse timbres and styles; (3) we provide controlled comparison and phoneme-level annotations of six commonly used singing techniques, helping technique modeling and control; (4) GTSinger offers realistic music scores, assisting real-world musical composition; (5) singing voices are accompanied by manual phoneme-to-audio alignments, global style labels, and 16.16 hours of paired speech for various singing tasks. Moreover, to facilitate the use of GTSinger, we conduct four benchmark experiments: technique-controllable singing voice synthesis, technique recognition, style transfer, and speech-to-singing conversion. The corpus and demos can be found at http://gtsinger.github.io.

Robert Downey Jr. praised Jon Favreau for being ambitious in his filmmaking, shouting out many films he has directed, including'The Lion King' and'The Jungle Book.' Robert Downey Jr. might be devoid of iron, but he's sure got some steel. The Academy Award-winning actor, 59, is speaking out about rapid technological advancements and how he plans to fight back if his name and likeness are manipulated by artificial intelligence. "I intend to sue," he told the "On with Kara Swisher" podcast. HOLLYWOOD EXECS WARN AI STEALS JOBS BUT CAN'T DO JOB OF TRUE ARTISTS: 'I WANT TO WORK WITH HUMAN BEINGS' Robert Downey Jr. says he plans to sue if someone manipulates his likeness through artificial intelligence. It all comes back to Downey Jr.'s alter ego, Tony Stark, whose own alter ego is Iron Man.

The Oscar-nominated actress is known for her roles in hit movies including Young Frankenstein and Tootsie.

Teri Garr, known for her work in "Young Frankenstein" and "Tootsie," died Tuesday in Los Angeles. Garr's publicist confirmed to The Associated Press that the comedian died of multiple sclerosis. She began her career in the entertainment industry as a background dancer in a number of Elvis Presley movies, and went on to earn an Academy Award nomination for her role as Sandy Lester in the 1982 Dustin Hoffman comedy, "Tootsie." Actress Teri Garr died Tuesday of multiple sclerosis. She was 79. (Getty Images) The daughter of Eddie Garr, a well-known vaudeville comedian and Phyllis Lind, one of the original Rockettes at New York's Radio City Music Hall, Garr seemed destined for show business.

What we once called the Google Chromecast (and then the Chromecast with Google TV) is now the Google TV Streamer. I won't pretend to understand the reasoning behind any product's rebrand, but at least this one makes a bit of sense. Casting content from elsewhere used to be a big reason TV dongles existed. Today, streaming devices primarily provide the brains required to watch content from Netflix, Disney and other streaming services on almost any screen, and casting is a bit of an afterthought. A name that focuses on Google TV's interface instead of casting seems right in 2024.

Welcome to our monthly digest, where you can catch up with any AIhub stories you may have missed, peruse the latest news, recap recent events, and more. This month, we learn about research towards safe and reliable AI agent behaviour, discuss generative AI hype, congratulate the Nobel Prize winners in physics and chemistry, and take a tour of recent conferences. In the latest in our series of interviews featuring the AAAI/ACM SIGAI doctoral consortium participants, we heard from Pulkit Verma about his research on safe and reliable behavior of AI agents. He is currently investigating the minimal set of requirements in an AI system that would enable a user to assess and understand the limits of its safe operability. There has been a string of articles recently about the end of generative AI hype.

Personalization of Large Language Models (LLMs) has recently become increasingly important with a wide range of applications. Despite the importance and recent progress, most existing works on personalized LLMs have focused either entirely on (a) personalized text generation or (b) leveraging LLMs for personalization-related downstream applications, such as recommendation systems. In this work, we bridge the gap between these two separate main directions for the first time by introducing a taxonomy for personalized LLM usage and summarizing the key differences and challenges. We provide a formalization of the foundations of personalized LLMs that consolidates and expands notions of personalization of LLMs, defining and discussing novel facets of personalization, usage, and desiderata of personalized LLMs. We then unify the literature across these diverse fields and usage scenarios by proposing systematic taxonomies for the granularity of personalization, personalization techniques, datasets, evaluation methods, and applications of personalized LLMs. Finally, we highlight challenges and important open problems that remain to be addressed. By unifying and surveying recent research using the proposed taxonomies, we aim to provide a clear guide to the existing literature and different facets of personalization in LLMs, empowering both researchers and practitioners.

The widespread and diverse online media platforms and other internet-driven communication technologies have presented significant challenges in defining the boundaries of freedom of expression. Consequently, the internet has been transformed into a potential cyber weapon. Within this evolving landscape, two particularly hazardous phenomena have emerged: fake news and doxxing. Although these threats have been subjects of extensive scholarly analysis, the crossroads where they intersect remain unexplored. This research addresses this convergence by introducing a novel system. The Fake News and Doxxing Detection with Explainable Artificial Intelligence (FNDEX) system leverages the capabilities of three distinct transformer models to achieve high-performance detection for both fake news and doxxing. To enhance data security, a rigorous three-step anonymization process is employed, rooted in a pattern-based approach for anonymizing personally identifiable information. Finally, this research emphasizes the importance of generating coherent explanations for the outcomes produced by both detection models. Our experiments on realistic datasets demonstrate that our system significantly outperforms the existing baselines