We propose a novel persona-driven data synthesis methodology that leverages various perspectives within a large language model (LLM) to create diverse synthetic data. To fully exploit this methodology at scale, we introduce Persona Hub -- a collection of 1 billion diverse personas automatically curated from web data. These 1 billion personas (~13% of the world's total population), acting as distributed carriers of world knowledge, can tap into almost every perspective encapsulated within the LLM, thereby facilitating the creation of diverse synthetic data at scale for various scenarios. By showcasing Persona Hub's use cases in synthesizing high-quality mathematical and logical reasoning problems, instructions (i.e., user prompts), knowledge-rich texts, game NPCs and tools (functions) at scale, we demonstrate persona-driven data synthesis is versatile, scalable, flexible, and easy to use, potentially driving a paradigm shift in synthetic data creation and applications in practice, which may have a profound impact on LLM research and development.

Recent work on evaluating the diversity of text generated by LLMs has focused on word-level features. Here we offer an analysis of syntactic features to characterize general repetition in models, beyond frequent n-grams. Specifically, we define syntactic templates and show that models tend to produce templated text in downstream tasks at a higher rate than what is found in human-reference texts. We find that most (76%) templates in model-generated text can be found in pre-training data (compared to only 35% of human-authored text), and are not overwritten during fine-tuning processes such as RLHF. This connection to the pre-training data allows us to analyze syntactic templates in models where we do not have the pre-training data. We also find that templates as features are able to differentiate between models, tasks, and domains, and are useful for qualitatively evaluating common model constructions. Finally, we demonstrate the use of templates as a useful tool for analyzing style memorization of training data in LLMs.

Background Dynamic graphs (DGs) are ubiquitous data structures present in various realworld evolving systems, such as social networks [1], linguistics [2], international relations [3], and computational finance [4]. Representing these dynamic graphs efficiently has become a crucial challenge due to their massive sizes and ever-changing nature. One compelling approach to tackle this challenge is discrete-time dynamic graph (DTDG) models [5-7], which represent a dynamic graph as a series of snapshots, each containing the nodes and edges that co-occur at particular timestamps. Despite the effectiveness of DTDG models in a wide range of graph-oriented tasks such as link prediction, node classification, and edge regression, these models usually remain opaque to researchers in terms of interpretability. The high-dimensional representations generated by these models make it difficult for users to extract and understand the intrinsic value from dynamic graphs. Currently, researchers often manually analyze the dynamic graph data, as there are no specialized tools to support this process [8, 9]. However, manual analysis of enormous dynamic graphs covering multiple timestamps can be overwhelming, and the continuously evolving nature of these graphs makes it challenging to intuitively capture both micro-level and macro-level structural shifts. For instance, in the study of international relations, aside from predicting graph attributes like future bilateral trade volumes, it is vital to understand microlevel changes such as a country's alliance network, trade relations, and conflict dynamics, as 1

Natural Language Inference (NLI) remains an important benchmark task for LLMs. NLI datasets are a springboard for transfer learning to other semantic tasks, and NLI models are standard tools for identifying the faithfulness of model-generated text. There are several large scale NLI datasets today, and models have improved greatly by hill-climbing on these collections. Yet their realistic performance on out-of-distribution/domain data is less well-understood. We explore the opportunity for synthetic high-quality datasets to adapt NLI models for zero-shot use in downstream applications across new and unseen text domains. We demonstrate a new approach for generating NLI data in diverse domains and lengths, so far not covered by existing training sets. The resulting examples have meaningful premises, the hypotheses are formed in creative ways rather than simple edits to a few premise tokens, and the labels have high accuracy. We show that models trained on this data ($685$K synthetic examples) have the best generalization to completely new downstream test settings. On the TRUE benchmark, a T5-small model trained with our data improves around $7\%$ on average compared to training on the best alternative dataset. The improvements are more pronounced for smaller models, while still meaningful on a T5 XXL model. We also demonstrate gains on test sets when in-domain training data is augmented with our domain-general synthetic data.

Large Language Models (LLMs) exhibit strong generalization capabilities to novel tasks when prompted with language instructions and in-context demos. Since this ability sensitively depends on the quality of prompts, various methods have been explored to automate the instruction design. While these methods demonstrated promising results, they also restricted the searched prompt to one instruction. Such simplification significantly limits their capacity, as a single demo-free instruction might not be able to cover the entire complex problem space of the targeted task. To alleviate this issue, we adopt the Mixture-of-Expert paradigm and divide the problem space into a set of sub-regions; Each sub-region is governed by a specialized expert, equipped with both an instruction and a set of demos. A two-phase process is developed to construct the specialized expert for each region: (1) demo assignment: Inspired by the theoretical connection between in-context learning and kernel regression, we group demos into experts based on their semantic similarity; (2) instruction assignment: A region-based joint search of an instruction per expert complements the demos assigned to it, yielding a synergistic effect. The resulting method, codenamed Mixture-of-Prompts (MoP), achieves an average win rate of 81% against prior arts across several major benchmarks.

There's a war brewing in Hollywood and we're not talking about how AI will inevitably kill us all by plagiarizing The Joker's chaos plans from The Dark Knight. The latest shot came from Dune director Denis Villeneuve in a red carpet interview in which he called the Wolverine & Deadpool popcorn bucket "horrific" and called the Dune buckets "unmatchable." Villeneuve did an impromptu interview with eTalkCTV where a reporter asked him about the feud that's been brewing between him and Deadpool star Ryan Reynolds over their respective popcorn receptacles. The reporter showed Villeneuve a picture of the Deadpool & Wolverine bucket featuring the yellow Wolverine's head and his gaping maw full of some of Orville Redenbacher's finest. Villeneuve said he doesn't have anything against the bucket but he thinks they are just riding the coattails he unfurled when the Dune sandworm popcorn bucket blew up the Internet.

The Center for Investigative Reporting, the nation's oldest nonprofit newsroom that produces Mother Jones and Reveal sued OpenAI and Microsoft in federal court on Thursday for allegedly using its content to train AI models without consent or compensation. "OpenAI and Microsoft started vacuuming up our stories to make their product more powerful, but they never asked for permission or offered compensation, unlike other organizations that license our material," said Monika Bauerlein, CEO of the Center for Investigative Reporting, in a statement. The work of journalists, at CIR and everywhere, is valuable, and OpenAI and Microsoft know it." Bauerlein said that OpenAI and Microsoft treat the work of nonprofit and independent publishers "as free raw material for their products," and added that such moves by generative AI companies hurt the public's access to truthful information in a "disappearing news landscape." OpenAI and Microsoft did not respond to a request for comment by Engadget.

Time has joined a growing number of publications to sign a licensing deal with OpenAI. The ChatGPT creator will legally be able to train its large language models on 101 years worth of the storied publication's journalism, as Axios first reported. OpenAI will also have access to real-time content from Time, with the apparent aim of answering user queries about breaking news. In return, OpenAI will cite Time and link back to source material on the publication's website. Perhaps Time will get a monetary kickback too, like other publishers that have shuffled over to OpenAI with a ragged cap in hand and an eye on one a new revenue source for struggling media companies.

YouTube is allegedly taking steps to avoid this issue, offering major musical labels payment to license their songs for AI training, the Financial Times reports. Sony Music Entertainment, Universal Music Group and Warner Records are all reportedly involved in talks with the Google-owned platform. However, it's unlikely the companies will get the last word as it would reportedly be up to each artist whether they participate. Many musicians are far from thrilled about allowing AI anywhere near their work. In April 2023, over 200 artists signed an open letter stating, "We must protect against the predatory use of AI to steal professional artists' voices and likenesses, violate creators' rights, and destroy the music ecosystem."

Our study delves into Multimodal Entity Linking, aligning the mention in multimodal information with entities in knowledge base. Existing methods are still facing challenges like ambiguous entity representations and limited image information utilization. Thus, we propose dynamic entity extraction using ChatGPT, which dynamically extracts entities and enhances datasets. We also propose a method: Dynamically Integrate Multimodal information with knowledge base (DIM), employing the capability of the Large Language Model (LLM) for visual understanding. The LLM, such as BLIP-2, extracts information relevant to entities in the image, which can facilitate improved extraction of entity features and linking them with the dynamic entity representations provided by ChatGPT. The experiments demonstrate that our proposed DIM method outperforms the majority of existing methods on the three original datasets, and achieves state-of-the-art (SOTA) on the dynamically enhanced datasets (Wiki+, Rich+, Diverse+).