Media
Adaptive Sampling-based Particle Filter for Visual-inertial Gimbal in the Wild
Kang, Xueyang, Herrera, Ariel, Lema, Henry, Valencia, Esteban, Vandewalle, Patrick
In this paper, we present a Computer Vision (CV) based tracking and fusion algorithm, dedicated to a 3D printed gimbal system on drones operating in nature. The whole gimbal system can stabilize the camera orientation robustly in a challenging nature scenario by using skyline and ground plane as references. Our main contributions are the following: a) a light-weight Resnet-18 backbone network model was trained from scratch, and deployed onto the Jetson Nano platform to segment the image into binary parts (ground and sky); b) our geometry assumption from nature cues delivers the potential for robust visual tracking by using the skyline and ground plane as a reference; c) a spherical surface-based adaptive particle sampling, can fuse orientation from multiple sensor sources flexibly. The whole algorithm pipeline is tested on our customized gimbal module including Jetson and other hardware components. The experiments were performed on top of a building in the real landscape.
An FAQ from the future -- how we struggled and defeated deepfakes
This one went smoothly -- no claims of rampant rigging, no significant taint of skulduggery -- due in large part to the defeat of deepfakes, democracy's newest enemy. Is such a future possible? So far, neither government nor the tech industry has agreed on effective guardrails against deepfakes. But this FAQ (from five years in the future) shows that the events of 2024 may well force the issue -- and that a solution is possible. Why did it take so long to find an effective way to fight deepfakes?
Grand Canyon record set by 92-year-old after months of training
Alfredo Aliaga Burdio, 92, set a Guinness World Record when he made a 24-mile hike across the Grand Canyon last October. A 92-year-old man is making headlines and setting records after he successfully took on a nearly 24-mile hike across the Grand Canyon in Arizona. Alfredo Aliaga Burdio, who currently resides in Berlin, completed his record-setting trek across the Grand Canyon on Oct. 15, 2023. That journey led to Burdio claiming the title of oldest person to cross the Grand Canyon rim-to-rim on foot (male), according to an announcement on New Year's Day by the Guinness World Records. Burdio's journey, which lasted for a total of 34 hours and 2 minutes, included 21 hours and 15 minutes of actual hiking time.
ChatGPT for Conversational Recommendation: Refining Recommendations by Reprompting with Feedback
Spurlock, Kyle Dylan, Acun, Cagla, Saka, Esin, Nasraoui, Olfa
Recommendation algorithms have been pivotal in handling the overwhelming volume of online content. However, these algorithms seldom consider direct user input, resulting in superficial interaction between them. Efforts have been made to include the user directly in the recommendation process through conversation, but these systems too have had limited interactivity. Recently, Large Language Models (LLMs) like ChatGPT have gained popularity due to their ease of use and their ability to adapt dynamically to various tasks while responding to feedback. In this paper, we investigate the effectiveness of ChatGPT as a top-n conversational recommendation system. We build a rigorous pipeline around ChatGPT to simulate how a user might realistically probe the model for recommendations: by first instructing and then reprompting with feedback to refine a set of recommendations. We further explore the effect of popularity bias in ChatGPT's recommendations, and compare its performance to baseline models. We find that reprompting ChatGPT with feedback is an effective strategy to improve recommendation relevancy, and that popularity bias can be mitigated through prompt engineering.
Few-Shot Causal Representation Learning for Out-of-Distribution Generalization on Heterogeneous Graphs
Ding, Pengfei, Wang, Yan, Liu, Guanfeng, Wang, Nan
Heterogeneous graph few-shot learning (HGFL) has been developed to address the label sparsity issue in heterogeneous graphs (HGs), which consist of various types of nodes and edges. The core concept of HGFL is to extract knowledge from rich-labeled classes in a source HG, transfer this knowledge to a target HG to facilitate learning new classes with few-labeled training data, and finally make predictions on unlabeled testing data. Existing methods typically assume that the source HG, training data, and testing data all share the same distribution. However, in practice, distribution shifts among these three types of data are inevitable due to two reasons: (1) the limited availability of the source HG that matches the target HG distribution, and (2) the unpredictable data generation mechanism of the target HG. Such distribution shifts result in ineffective knowledge transfer and poor learning performance in existing methods, thereby leading to a novel problem of out-of-distribution (OOD) generalization in HGFL. To address this challenging problem, we propose a novel Causal OOD Heterogeneous graph Few-shot learning model, namely COHF. In COHF, we first characterize distribution shifts in HGs with a structural causal model, establishing an invariance principle for OOD generalization in HGFL. Then, following this invariance principle, we propose a new variational autoencoder-based heterogeneous graph neural network to mitigate the impact of distribution shifts. Finally, by integrating this network with a novel meta-learning framework, COHF effectively transfers knowledge to the target HG to predict new classes with few-labeled data. Extensive experiments on seven real-world datasets have demonstrated the superior performance of COHF over the state-of-the-art methods.
Building Efficient and Effective OpenQA Systems for Low-Resource Languages
Budur, Emrah, Özçelik, Rıza, Soylu, Dilara, Khattab, Omar, Güngör, Tunga, Potts, Christopher
Question answering (QA) is the task of answering questions posed in natural language with free-form natural language answers extracted from a given passage. In the OpenQA variant, only a question text is given, and the system must retrieve relevant passages from an unstructured knowledge source and use them to provide answers, which is the case in the mainstream QA systems on the Web. QA systems currently are mostly limited to the English language due to the lack of large-scale labeled QA datasets in non-English languages. In this paper, we show that effective, low-cost OpenQA systems can be developed for low-resource languages. The key ingredients are (1) weak supervision using machine-translated labeled datasets and (2) a relevant unstructured knowledge source in the target language. Furthermore, we show that only a few hundred gold assessment examples are needed to reliably evaluate these systems. We apply our method to Turkish as a challenging case study, since English and Turkish are typologically very distinct. We present SQuAD-TR, a machine translation of SQuAD2.0, and we build our OpenQA system by adapting ColBERT-QA for Turkish. We obtain a performance improvement of 9-34% in the EM score and 13-33% in the F1 score compared to the BM25-based and DPR-based baseline QA reader models by using two versions of Wikipedia dumps spanning two years. Our results show that SQuAD-TR makes OpenQA feasible for Turkish, which we hope encourages researchers to build OpenQA systems in other low-resource languages. We make all the code, models, and the dataset publicly available.
Maintaining Journalistic Integrity in the Digital Age: A Comprehensive NLP Framework for Evaluating Online News Content
Bojic, Ljubisa, Prodanovic, Nikola, Samala, Agariadne Dwinggo
The rapid growth of online news platforms has led to an increased need for reliable methods to evaluate the quality and credibility of news articles. This paper proposes a comprehensive framework to analyze online news texts using natural language processing (NLP) techniques, particularly a language model specifically trained for this purpose, alongside other well-established NLP methods. The framework incorporates ten journalism standards-objectivity, balance and fairness, readability and clarity, sensationalism and clickbait, ethical considerations, public interest and value, source credibility, relevance and timeliness, factual accuracy, and attribution and transparency-to assess the quality of news articles. By establishing these standards, researchers, media organizations, and readers can better evaluate and understand the content they consume and produce. The proposed method has some limitations, such as potential difficulty in detecting subtle biases and the need for continuous updating of the language model to keep pace with evolving language patterns.
Amplification of Addictive New Media Features in the Metaverse
Bojic, Ljubisa, Matthes, Joerg, Cabarkapa, Milan
The emergence of the metaverse, envisioned as a hyperreal virtual universe facilitating boundless human interaction, stands to revolutionize our conception of media, with significant impacts on addiction, creativity, relationships, and social polarization. This paper aims to dissect the addictive potential of the metaverse due to its immersive and interactive features, scrutinize the effects of its recommender systems on creativity and social polarization, and explore potential consequences stemming from the metaverse development. We employed a literature review methodology, drawing parallels from the research on new media platforms and examining the progression of reality-mimicking features in media from historical perspectives to understand this transformative digital frontier. The findings suggest that these immersive and interactive features could potentially exacerbate media addiction. The designed recommender systems, while aiding personalization and user engagement, might contribute to social polarization and affect the diversity of creative output. However, our conclusions are based primarily on theoretical propositions from studies conducted on existing media platforms and lack empirical support specific to the metaverse. Therefore, this paper identifies a critical gap requiring further research, through empirical studies focused on metaverse use and addiction and exploration of privacy, security, and ethical implications associated with this burgeoning digital universe. As the development of the metaverse accelerates, it is incumbent on scholars, technologists, and policymakers to navigate its multilayered impacts thoughtfully to balance innovation with societal well-being.
MLCA-AVSR: Multi-Layer Cross Attention Fusion based Audio-Visual Speech Recognition
Wang, He, Guo, Pengcheng, Zhou, Pan, Xie, Lei
While automatic speech recognition (ASR) systems degrade significantly Following this, plenty of studies have adopted a cross-attention module in noisy environments, audio-visual speech recognition to capture inherent alignments and complementary information (AVSR) systems aim to complement the audio stream with noiseinvariant between fully encoded audio-visual representations [9, 10, 11]. Additionally, visual cues and improve the system's robustness. However, some works directly concatenate the raw speech and video current studies mainly focus on fusing the well-learned modality sequences together and employ a shared encoder with self-attention features, like the output of modality-specific encoders, without mechanisms to learn modality alignments [2, 12]. In [13, 14], considering the contextual relationship during the modality feature hidden features from different layers of audio and visual encoders learning. In this study, we propose a multi-layer cross-attention were leveraged to achieve more effective fusion, indicating that conducting fusion based AVSR (MLCA-AVSR) approach that promotes representation multi-layer fusion can promote the performance of AVSR learning of each modality by fusing them at different levels systems. of audio/visual encoders. Experimental results on the MISP2022-Recently, the Multi-modal Information based Speech Processing AVSR Challenge dataset show the efficacy of our proposed system, (MISP) Challenge series [15, 16, 17] has been introduced to achieving a concatenated minimum permutation character error rate explore the utilization of both audio and visual data in distant multimicrophone (cpCER) of 30.57% on the Eval set and yielding up to 3.17% relative signal processing tasks, like keyword spotting and improvement compared with our previous system which ranked speech recognition.
Pragmatic Evaluation of Clarifying Questions with Fact-Level Masking
Toles, Matthew, Huang, Yukun, Yu, Zhou, Gravano, Luis
The ability to derive useful information by asking clarifying questions (ACQ) is an important element of real life collaboration on reasoning tasks, such as question answering (QA). Existing natural language ACQ challenges, however, evaluate generations based on word overlap rather than the value of the information itself. Word overlap is often an inappropriate metric for question generation since many different questions could be useful in a given situation, and a single question can be phrased many different ways. Instead, we propose evaluating questions pragmatically based on the value of the information they retrieve. Here we present a definition and framework for natural language pragmatic asking of clarifying questions (PACQ), the problem of generating questions that result in answers useful for a reasoning task. We also present fact-level masking (FLM), a procedure for converting natural language datasets into self-supervised PACQ datasets by omitting particular critical facts. Finally, we generate a PACQ dataset from the HotpotQA dataset using FLM and evaluate several zero-shot language models on it. Our experiments show that current zero-shot models struggle to ask questions that retrieve useful information, as compared to human annotators. These results demonstrate an opportunity to use FLM datasets and the PACQ framework to objectively evaluate and improve question generation and other language models.