Media
Natural Language Generation for Visualizations: State of the Art, Challenges and Future Directions
Hoque, Enamul, Islam, Mohammed Saidul
Natural language and visualization are two complementary modalities of human communication that play a crucial role in conveying information effectively. While visualizations help people discover trends, patterns, and anomalies in data, natural language descriptions help explain these insights. Thus, combining text with visualizations is a prevalent technique for effectively delivering the core message of the data. Given the rise of natural language generation (NLG), there is a growing interest in automatically creating natural language descriptions for visualizations, which can be used as chart captions, answering questions about charts, or telling data-driven stories. In this survey, we systematically review the state of the art on NLG for visualizations and introduce a taxonomy of the problem. The NLG tasks fall within the domain of Natural Language Interfaces (NLI) for visualization, an area that has garnered significant attention from both the research community and industry. To narrow down the scope of the survey, we primarily concentrate on the research works that focus on text generation for visualizations. To characterize the NLG problem and the design space of proposed solutions, we pose five Wh-questions, why and how NLG tasks are performed for visualizations, what the task inputs and outputs are, as well as where and when the generated texts are integrated with visualizations. We categorize the solutions used in the surveyed papers based on these "five Wh-questions." Finally, we discuss the key challenges and potential avenues for future research in this domain.
Instruction Embedding: Latent Representations of Instructions Towards Task Identification
Li, Yiwei, Shi, Jiayi, Feng, Shaoxiong, Yuan, Peiwen, Wang, Xinglin, Pan, Boyuan, Wang, Heda, Hu, Yao, Li, Kan
Instruction data is crucial for improving the capability of Large Language Models (LLMs) to align with human-level performance. Recent research LIMA demonstrates that alignment is essentially a process where the model adapts instructions' interaction style or format to solve various tasks, leveraging pre-trained knowledge and skills. Therefore, for instructional data, the most important aspect is the task it represents, rather than the specific semantics and knowledge information. The latent representations of instructions play roles for some instruction-related tasks like data selection and demonstrations retrieval. However, they are always derived from text embeddings, encompass overall semantic information that influences the representation of task categories. In this work, we introduce a new concept, instruction embedding, and construct Instruction Embedding Benchmark (IEB) for its training and evaluation. Then, we propose a baseline Prompt-based Instruction Embedding (PIE) method to make the representations more attention on tasks. The evaluation of PIE, alongside other embedding methods on IEB with two designed tasks, demonstrates its superior performance in accurately identifying task categories.
Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMs
Zeng, Fengzhu, Li, Wenqian, Gao, Wei, Pang, Yan
Detecting multimodal misinformation, especially in the form of image-text pairs, is crucial. Obtaining large-scale, high-quality real-world fact-checking datasets for training detectors is costly, leading researchers to use synthetic datasets generated by AI technologies. However, the generalizability of detectors trained on synthetic data to real-world scenarios remains unclear due to the distribution gap. To address this, we propose learning from synthetic data for detecting real-world multimodal misinformation through two model-agnostic data selection methods that match synthetic and real-world data distributions. Experiments show that our method enhances the performance of a small MLLM (13B) on real-world fact-checking datasets, enabling it to even surpass GPT-4V~\cite{GPT-4V}.
Here's a peek at how A Minecraft Movie will handle crafting
The team behind the upcoming Minecraft movie shared a new clip during Minecraft Live that expands on the brief crafting moment we saw in the polarizing first teaser. The scene comes in the middle of a discussion between Mojang creative director Torfi Frans Olafsson and A Minecraft Movie director Jared Hess, at 4:51. The segment also gives us our first look at the movie's interpretation of a Minecraft bee, which I'm not quite sure how to feel about yet. That you can find toward the end of the video. A Minecraft Movie is slated for release in April 2025 and stars Jack Black as Steve, alongside Jason Momoa, Danielle Brooks, Emma Myers and Sebastian Eugene Hansen.
Public interest in science or bots? Selective amplification of scientific articles on Twitter
Rahman, Ashiqur, Mohammadi, Ehsan, Alhoori, Hamed
With the remarkable capability to reach the public instantly, social media has become integral in sharing scholarly articles to measure public response. Since spamming by bots on social media can steer the conversation and present a false public interest in given research, affecting policies impacting the public's lives in the real world, this topic warrants critical study and attention. We used the Altmetric dataset in combination with data collected through the Twitter Application Programming Interface (API) and the Botometer API. We combined the data into an extensive dataset with academic articles, several features from the article and a label indicating whether the article had excessive bot activity on Twitter or not. We analyzed the data to see the possibility of bot activity based on different characteristics of the article. We also trained machine-learning models using this dataset to identify possible bot activity in any given article. Our machine-learning models were capable of identifying possible bot activity in any academic article with an accuracy of 0.70. We also found that articles related to "Health and Human Science" are more prone to bot activity compared to other research areas. Without arguing the maliciousness of the bot activity, our work presents a tool to identify the presence of bot activity in the dissemination of an academic article and creates a baseline for future research in this direction.
Food at college gets high-tech boost with first robotic kitchen in university setting
Denisse Castillo, senior director of residential dining at Florida International University, describes what it's like working with the first robotic kitchen in a campus setting. Hungry students at Florida International University (FIU) near Miami are being fed by a robot these days. "Beastro" – yes, it has a name – is the first robotic kitchen in the country to be used in a university setting, according to FIU. (See the video at the top of this article, and another one down below.) On a recent morning at the Ernest R. Graham University Center on FIU's flagship campus, Beastro prepared chicken teriyaki for Jocelyn Hernandez, 22, a senior studying natural and applied sciences, as Fox News Digital watched and filmed. Soon after, Beastro was busy making a cheese omelet for Pablo Reyes, 20, a junior biomedical engineering student.
The Guide #158: Video games are the new frontier for pop culture's obsession with the past
The past is a big deal in the video games industry right now. Hardly a month goes by when we're not being tempted by a new retro mini console, whether that's a cutesy Nintendo or a demure ZX Spectrum (a new version of which is arriving in November, complete with rubbery keys and 48 legendary games). And this year's release schedule is absolutely crammed with remasters of classic titles. In April, the video game news site Kotaku listed 30 old timers being exhumed and revived for 2024, including The Last of Us Part II, Tomb Raider 1-3 and Star Wars: Dark Forces. And the article missed a few! October alone will see updated versions of horror adventures Until Dawn, Silent Hill 2 and Clock Tower, as well as Lego Harry Potter.
China's Plan to Make AI Watermarks Happen
These are some of the things the Chinese government wants AI companies and social media platforms to use to properly label AI-generated content and crack down against misinformation. On September 14, China's Cyberspace Administration drafted a new regulation that aims to inform people of whether something is real or AI. As generative AI tools get increasingly advanced, the difficulty to discern whether content is AI-generated is causing all kinds of serious issues, from nonconsensual porn to political disinformation. China's is not the first regime to tackle this issue--the European Union's AI Act, adopted this March, also requires similar labels; California passed a similar bill this month. And China's previous AI regulations also briefly mentioned the need for gen-AI labels. However, this new policy outlines more details of how AI watermarks should be implemented by platforms.
'CHiPs' star Erik Estrada says certain people using AI are not 'very Christian'
"CHiPs" star Erik Estrada shared a warning about how artificial intelligence can "destroy lives." During an interview with Fox News Digital, the 75-year-old actor and "Divine Renovation" host acknowledged the benefits of AI but cautioned that the new technology is also frequently being used for nefarious purposes. "I think just like the Internet, just like the cell phones, just like everything -- they need to just use the positive side of it," Estrada said. "The side which can help or employ and create goodwill, good things, good jobs, good fortune for people that want to go in that direction and not, of course, use the negative stuff." "CHiPs" star Erik Estrada warned about the dangers posed by AI. (Brian To/FilmMagic) Estrada pointed to how AI can be used to create deepfakes -- deceptive pictures, videos and audio that misrepresent people or events.
BeanCounter: A low-toxicity, large-scale, and open dataset of business-oriented text
Many of the recent breakthroughs in language modeling have resulted from scaling effectively the same model architecture to larger datasets. In this vein, recent work has highlighted performance gains from increasing training dataset size and quality, suggesting a need for novel sources of large-scale datasets. In this work, we introduce BeanCounter, a public dataset consisting of more than 159B tokens extracted from businesses' disclosures. We show that this data is indeed novel: less than 0.1% of BeanCounter appears in Common Crawl-based datasets and it is an order of magnitude larger than datasets relying on similar sources. Given the data's provenance, we hypothesize that BeanCounter is comparatively more factual and less toxic than web-based datasets. Exploring this hypothesis, we find that many demographic identities occur with similar prevalence in BeanCounter but with significantly less toxic context relative to other datasets. To demonstrate the utility of BeanCounter, we evaluate and compare two LLMs continually pre-trained on BeanCounter with their base models. We find an 18-33% reduction in toxic generation and improved performance within the finance domain for the continually pretrained models. Collectively, our work suggests that BeanCounter is a novel source of low-toxicity and high-quality domain-specific data with sufficient scale to train multi-billion parameter LLMs.