Goto

Collaborating Authors

 Media


How weather apps are trying to be more accurate

BBC News

We have global audience with over 100 languages and dialects, so all these different products have to be converted into number of different languages which can be consumed by users,


MUFFIN: Curating Multi-Faceted Instructions for Improving Instruction-Following

arXiv.org Artificial Intelligence

In the realm of large language models (LLMs), enhancing instruction-following capability often involves curating expansive training data. This is achieved through two primary schemes: i) Scaling-Inputs: Amplifying (input, output) pairs per task instruction, aiming for better instruction adherence. ii) Scaling Input-Free Tasks: Enlarging tasks, each composed of an (instruction, output) pair (without requiring a separate input anymore). However, LLMs under Scaling-Inputs tend to be overly sensitive to inputs, leading to misinterpretation or non-compliance with instructions. Conversely, Scaling Input-Free Tasks demands a substantial number of tasks but is less effective in instruction following when dealing with instances in Scaling-Inputs. This work introduces MUFFIN, a new scheme of instruction-following dataset curation. Specifically, we automatically Scale Tasks per Input by diversifying these tasks with various input facets. Experimental results across four zero-shot benchmarks, spanning both Scaling-Inputs and Scaling Input-Free Tasks schemes, reveal that LLMs, at various scales, trained on MUFFIN generally demonstrate superior instruction-following capabilities compared to those trained on the two aforementioned schemes.


On the Trade-Off between Stability and Representational Capacity in Graph Neural Networks

arXiv.org Artificial Intelligence

Analyzing the stability of graph neural networks (GNNs) under topological perturbations is key to understanding their transferability and the role of each architecture component. However, stability has been investigated only for particular architectures, questioning whether it holds for a broader spectrum of GNNs or only for a few instances. To answer this question, we study the stability of EdgeNet: a general GNN framework that unifies more than twenty solutions including the convolutional and attention-based classes, as well as graph isomorphism networks and hybrid architectures. We prove that all GNNs within the EdgeNet framework are stable to topological perturbations. By studying the effect of different EdgeNet categories on the stability, we show that GNNs with fewer degrees of freedom in their parameter space, linked to a lower representational capacity, are more stable. The key factor yielding this trade-off is the eigenvector misalignment between the EdgeNet parameter matrices and the graph shift operator. For example, graph convolutional neural networks that assign a single scalar per signal shift (hence, with a perfect alignment) are more stable than the more involved node or edge-varying counterparts. Extensive numerical results corroborate our theoretical findings and highlight the role of different architecture components in the trade-off.


Generative Powers of Ten

arXiv.org Artificial Intelligence

We present a method that uses a text-to-image model to generate consistent content across multiple image scales, enabling extreme semantic zooms into a scene, e.g., ranging from a wide-angle landscape view of a forest to a macro shot of an insect sitting on one of the tree branches. We achieve this through a joint multi-scale diffusion sampling approach that encourages consistency across different scales while preserving the integrity of each individual sampling process. Since each generated scale is guided by a different text prompt, our method enables deeper levels of zoom than traditional super-resolution methods that may struggle to create new contextual structure at vastly different scales. We compare our method qualitatively with alternative techniques in image super-resolution and outpainting, and show that our method is most effective at generating consistent multi-scale content.


When it Rains, it Pours: Modeling Media Storms and the News Ecosystem

arXiv.org Artificial Intelligence

Most events in the world receive at most brief coverage by the news media. Occasionally, however, an event will trigger a media storm, with voluminous and widespread coverage lasting for weeks instead of days. In this work, we develop and apply a pairwise article similarity model, allowing us to identify story clusters in corpora covering local and national online news, and thereby create a comprehensive corpus of media storms over a nearly two year period. Using this corpus, we investigate media storms at a new level of granularity, allowing us to validate claims about storm evolution and topical distribution, and provide empirical support for previously hypothesized patterns of influence of storms on media coverage and intermedia agenda setting.


TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding

arXiv.org Artificial Intelligence

This work proposes TimeChat, a time-sensitive multimodal large language model specifically designed for long video understanding. Our model incorporates two key architectural contributions: (1) a timestamp-aware frame encoder that binds visual content with the timestamp of each frame, and (2) a sliding video Q-Former that produces a video token sequence of varying lengths to accommodate videos of various durations. Additionally, we construct an instruction-tuning dataset, encompassing 6 tasks and a total of 125K instances, to further enhance TimeChat's instruction-following performance. Experiment results across various video understanding tasks, such as dense captioning, temporal grounding, and highlight detection, demonstrate TimeChat's strong zero-shot temporal localization and reasoning capabilities. For example, it achieves +9.2 F1 score and +2.8 CIDEr on YouCook2, +5.8 HIT@1 on QVHighlights, and +27.5 R@1 (IoU=0.5) on Charades-STA, compared to state-of-the-art video large language models, holding the potential to serve as a versatile video assistant for long-form video comprehension tasks and satisfy realistic user requirements.


Distilled Self-Critique of LLMs with Synthetic Data: a Bayesian Perspective

arXiv.org Artificial Intelligence

Review: No Country for Old Men is an extraordinary movie that seamlessly blends elements of crime, drama, and psychological suspense into a cohesive and awe-inspiring work of art. From the opening scene to the final heart-stopping moments, director Joel Cohen has crafted a visually stunning vision that both challenges and captivates the viewer. The cinematography is unparalleled in its ability to convey emotion and character without resorting to cheap tricks or manipulation. The cast members all deliver impressive performances that allow us to empathize with their characters while simultaneously questioning their motives. From Javier Bardem's chilling portrayal of the villain to Tommy Lee Jones' nuanced exploration of a man faced with an impossible moral dilemma. Despite its lengthy runtime, No Country for Old Men maintains an intense narrative that keeps audiences engaged until the very end.


Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model

arXiv.org Artificial Intelligence

Both graph structures and textual information play a critical role in Knowledge Graph Completion (KGC). With the success of Pre-trained Language Models (PLMs) such as BERT, they have been applied for text encoding for KGC. However, the current methods mostly prefer to fine-tune PLMs, leading to huge training costs and limited scalability to larger PLMs. In contrast, we propose to utilize prompts and perform KGC on a frozen PLM with only the prompts trained. Accordingly, we propose a new KGC method named PDKGC with two prompts -- a hard task prompt which is to adapt the KGC task to the PLM pre-training task of token prediction, and a disentangled structure prompt which learns disentangled graph representation so as to enable the PLM to combine more relevant structure knowledge with the text information. With the two prompts, PDKGC builds a textual predictor and a structural predictor, respectively, and their combination leads to more comprehensive entity prediction. Solid evaluation on two widely used KGC datasets has shown that PDKGC often outperforms the baselines including the state-of-the-art, and its components are all effective. Our codes and data are available at https://github.com/genggengcss/PDKGC.


STADEE: STAtistics-based DEEp Detection of Machine Generated Text

arXiv.org Artificial Intelligence

In recent years, there have been notable advancements in the field of natural language generation, particularly with the development of large-scale PLMs like ChatGPT [1] and GPT-4 [2]. The texts produced by these models are of such exceptional quality that it can be challenging for humans to discern them from those written by people. In fact, according to a technical report by OpenAI, the majority of texts generated by GPT-2 were already indistinguishable from those written by humans [3]. These PLMs have a broad range of applications, including story [4] and dialogue generation [5], as well as code writing [6]. Nonetheless, they can also be easily exploited by malicious actors to fabricate fake news [7, 8, 9] and comments [10] for personal profit or political interference, thereby posing a significant threat to society. Therefore, it is imperative to explore automatic methods for detecting machine-generated text to identify disinformation and mitigate the likelihood of abuse [11].


Customize your NeRF: Adaptive Source Driven 3D Scene Editing via Local-Global Iterative Training

arXiv.org Artificial Intelligence

In this paper, we target the adaptive source driven 3D scene editing task by proposing a CustomNeRF model that unifies a text description or a reference image as the editing prompt. However, obtaining desired editing results conformed with the editing prompt is nontrivial since there exist two significant challenges, including accurate editing of only foreground regions and multi-view consistency given a single-view reference image. To tackle the first challenge, we propose a Local-Global Iterative Editing (LGIE) training scheme that alternates between foreground region editing and full-image editing, aimed at foreground-only manipulation while preserving the background. For the second challenge, we also design a class-guided regularization that exploits class priors within the generation model to alleviate the inconsistency problem among different views in image-driven editing. Extensive experiments show that our CustomNeRF produces precise editing results under various real scenes for both text- and image-driven settings.