Media
WHAC: World-grounded Humans and Cameras
Yin, Wanqi, Cai, Zhongang, Wang, Ruisi, Wang, Fanzhou, Wei, Chen, Mei, Haiyi, Xiao, Weiye, Yang, Zhitao, Sun, Qingping, Yamashita, Atsushi, Liu, Ziwei, Yang, Lei
Estimating human and camera trajectories with accurate scale in the world coordinate system from a monocular video is a highly desirable yet challenging and ill-posed problem. In this study, we aim to recover expressive parametric human models (i.e., SMPL-X) and corresponding camera poses jointly, by leveraging the synergy between three critical players: the world, the human, and the camera. Our approach is founded on two key observations. Firstly, camera-frame SMPL-X estimation methods readily recover absolute human depth. Secondly, human motions inherently provide absolute spatial cues. By integrating these insights, we introduce a novel framework, referred to as WHAC, to facilitate world-grounded expressive human pose and shape estimation (EHPS) alongside camera pose estimation, without relying on traditional optimization techniques. Additionally, we present a new synthetic dataset, WHAC-A-Mole, which includes accurately annotated humans and cameras, and features diverse interactive human motions as well as realistic camera trajectories. Extensive experiments on both standard and newly established benchmarks highlight the superiority and efficacy of our framework. We will make the code and dataset publicly available.
CrossTune: Black-Box Few-Shot Classification with Label Enhancement
Luo, Danqing, Zhang, Chen, Zhang, Yan, Li, Haizhou
Training or finetuning large-scale language models (LLMs) requires substantial computation resources, motivating recent efforts to explore parameter-efficient adaptation to downstream tasks. One approach is to treat these models as black boxes and use forward passes (Inference APIs) to interact with them. Current research focuses on adapting these black-box models to downstream tasks using gradient-free prompt optimization, but this often involves an expensive process of searching task-specific prompts. Therefore, we are motivated to study black-box language model adaptation without prompt search. Specifically, we introduce a label-enhanced cross-attention network called CrossTune, which models the semantic relatedness between the input text sequence and task-specific label descriptions. Its effectiveness is examined in the context of few-shot text classification. To improve the generalization of CrossTune, we utilize ChatGPT to generate additional training data through in-context learning. A switch mechanism is implemented to exclude low-quality ChatGPT-generated data. Through extensive experiments on seven benchmark text classification datasets, we demonstrate that our proposed approach outperforms the previous state-of-the-art gradient-free black-box tuning method by 5.7% on average. Even without using ChatGPT-augmented data, CrossTune performs better or comparably than previous black-box tuning methods, suggesting the effectiveness of our approach.
InBox: Recommendation with Knowledge Graph using Interest Box Embedding
Xu, Zezhong, Qu, Yincen, Zhang, Wen, Liang, Lei, Chen, Huajun
Knowledge graphs (KGs) have become vitally important in modern recommender systems, effectively improving performance and interpretability. Fundamentally, recommender systems aim to identify user interests based on historical interactions and recommend suitable items. However, existing works overlook two key challenges: (1) an interest corresponds to a potentially large set of related items, and (2) the lack of explicit, fine-grained exploitation of KG information and interest connectivity. This leads to an inability to reflect distinctions between entities and interests when modeling them in a single way. Additionally, the granularity of concepts in the knowledge graphs used for recommendations tends to be coarse, failing to match the fine-grained nature of user interests. This homogenization limits the precise exploitation of knowledge graph data and interest connectivity. To address these limitations, we introduce a novel embedding-based model called InBox. Specifically, various knowledge graph entities and relations are embedded as points or boxes, while user interests are modeled as boxes encompassing interaction history. Representing interests as boxes enables containing collections of item points related to that interest. We further propose that an interest comprises diverse basic concepts, and box intersection naturally supports concept combination. Across three training steps, InBox significantly outperforms state-of-the-art methods like HAKG and KGIN on recommendation tasks. Further analysis provides meaningful insights into the variable value of different KG data for recommendations. In summary, InBox advances recommender systems through box-based interest and concept modeling for sophisticated knowledge graph exploitation.
TT-BLIP: Enhancing Fake News Detection Using BLIP and Tri-Transformer
Detecting fake news has received a lot of attention. Many previous methods concatenate independently encoded unimodal data, ignoring the benefits of integrated multimodal information. Also, the absence of specialized feature extraction for text and images further limits these methods. This paper introduces an end-to-end model called TT-BLIP that applies the bootstrapping language-image pretraining for unified vision-language understanding and generation (BLIP) for three types of information: BERT and BLIP\textsubscript{Txt} for text, ResNet and BLIP\textsubscript{Img} for images, and bidirectional BLIP encoders for multimodal information. The Multimodal Tri-Transformer fuses tri-modal features using three types of multi-head attention mechanisms, ensuring integrated modalities for enhanced representations and improved multimodal data analysis. The experiments are performed using two fake news datasets, Weibo and Gossipcop. The results indicate TT-BLIP outperforms the state-of-the-art models.
Optimal and Adaptive Non-Stationary Dueling Bandits Under a Generalized Borda Criterion
In dueling bandits, the learner receives preference feedback between arms, and the regret of an arm is defined in terms of its suboptimality to a winner arm. The more challenging and practically motivated non-stationary variant of dueling bandits, where preferences change over time, has been the focus of several recent works (Saha and Gupta, 2022; Buening and Saha, 2023; Suk and Agarwal, 2023). The goal is to design algorithms without foreknowledge of the amount of change. The bulk of known results here studies the Condorcet winner setting, where an arm preferred over any other exists at all times. Yet, such a winner may not exist and, to contrast, the Borda version of this problem (which is always well-defined) has received little attention. In this work, we establish the first optimal and adaptive Borda dynamic regret upper bound, which highlights fundamental differences in the learnability of severe non-stationarity between Condorcet vs. Borda regret objectives in dueling bandits. Surprisingly, our techniques for non-stationary Borda dueling bandits also yield improved rates within the Condorcet winner setting, and reveal new preference models where tighter notions of non-stationarity are adaptively learnable. This is accomplished through a novel generalized Borda score framework which unites the Borda and Condorcet problems, thus allowing reduction of Condorcet regret to a Borda-like task. Such a generalization was not previously known and is likely to be of independent interest.
"3 Body Problem" Is a Rare Species of Sci-Fi Epic
Early in "3 Body Problem," the new Netflix adaptation of Liu Cixin's acclaimed science-fiction trilogy, intelligent life from another corner of the universe decides that a spectacle is required to get humanity's attention. On a cloudless night, the stars brighten, then flicker on and off, as if a kid were playing with a light switch, transmitting a series of numbers. Two physicists--one high and thus mesmerized, the other terrified--watch the phenomenon from a Gothic courtyard in Oxford, England. The next day, the stoner, Saul Durand (Jovan Adepo), chalks the experience up to an elaborate hoax; the rest of the world also saw the stars twinkle in code, but the celestial blinks went undetected by Earth's most powerful telescopes. The otherworldly signal may have been a message just for Saul's companion, a nanomaterials researcher named Auggie Salazar (Eiza González) who's had a glowing countdown emblazoned across her field of vision for days.
House AI Task Force chairman eyes public and private hearings as lawmakers mull regulation
Rep. Jay Obernolte was selected to lead the House task force on AI. Fox News Digital speaks with the California Republican about his goals for the panel and his own thoughts about the rapidly advancing technology. EXCLUSIVE: The chairman of the House of Representatives' new AI Task Force said his panel will likely hold hearings on artificial intelligence as Congress seeks to get ahead of the rapidly advancing technology. "Our number one task is to, by the end of the year, issue a report that details a regulatory framework for artificial intelligence. That framework is going to have a number of different pillars. And those pillars will come out of the things that our task force members are concerned about," Rep. Jay Obernolte, R-Calif., told Fox News Digital.
South Korea-hosted summit warns of AI risks to democracy
South Korean President Yoon Suk-yeol on Monday called fake news and disinformation based on AI and digital technology threats to democracy, as some officials attending a global summit accused Russia and China of conducting malicious propaganda campaigns. Speaking at the opening of the Summit for Democracy being held in Seoul, Yoon said countries had a duty to share experiences and wisdom so that artificial intelligence and technology could be employed to promote democracy. "Fake news and disinformation based on artificial intelligence and digital technology not only violates individual freedom and human rights but also threatens democratic systems," Yoon said.
Generalized Multi-Source Inference for Text Conditioned Music Diffusion Models
Postolache, Emilian, Mariani, Giorgio, Cosmo, Luca, Benetos, Emmanouil, Rodolà, Emanuele
Multi-Source Diffusion Models (MSDM) allow for compositional musical generation tasks: generating a set of coherent sources, creating accompaniments, and performing source separation. Despite their versatility, they require estimating the joint distribution over the sources, necessitating pre-separated musical data, which is rarely available, and fixing the number and type of sources at training time. This paper generalizes MSDM to arbitrary time-domain diffusion models conditioned on text embeddings. These models do not require separated data as they are trained on mixtures, can parameterize an arbitrary number of sources, and allow for rich semantic control. We propose an inference procedure enabling the coherent generation of sources and accompaniments. Additionally, we adapt the Dirac separator of MSDM to perform source separation. We experiment with diffusion models trained on Slakh2100 and MTG-Jamendo, showcasing competitive generation and separation results in a relaxed data setting.
QEAN: Quaternion-Enhanced Attention Network for Visual Dance Generation
Zhou, Zhizhen, Huo, Yejing, Huang, Guoheng, Zeng, An, Chen, Xuhang, Huang, Lian, Li, Zinuo
The study of music-generated dance is a novel and challenging Image generation task. It aims to input a piece of music and seed motions, then generate natural dance movements for the subsequent music. Transformer-based methods face challenges in time series prediction tasks related to human movements and music due to their struggle in capturing the nonlinear relationship and temporal aspects. This can lead to issues like joint deformation, role deviation, floating, and inconsistencies in dance movements generated in response to the music. In this paper, we propose a Quaternion-Enhanced Attention Network (QEAN) for visual dance synthesis from a quaternion perspective, which consists of a Spin Position Embedding (SPE) module and a Quaternion Rotary Attention (QRA) module. First, SPE embeds position information into self-attention in a rotational manner, leading to better learning of features of movement sequences and audio sequences, and improved understanding of the connection between music and dance. Second, QRA represents and fuses 3D motion features and audio features in the form of a series of quaternions, enabling the model to better learn the temporal coordination of music and dance under the complex temporal cycle conditions of dance generation. Finally, we conducted experiments on the dataset AIST++, and the results show that our approach achieves better and more robust performance in generating accurate, high-quality dance movements. Our source code and dataset can be available from https://github.com/MarasyZZ/QEAN and https://google.github.io/aistplusplus_dataset respectively.