Large Language Model
Using Natural Language Processing to find Indication for Burnout with Text Classification: From Online Data to Real-World Data
Kurpicz-Briki, Mascha, Merhbene, Ghofrane, Puttick, Alexandre, Souissi, Souhir Ben, Bieri, Jannic, Müller, Thomas Jörg, Golz, Christoph
Burnout, classified as a syndrome in the ICD-11, arises from chronic workplace stress that has not been effectively managed. It is characterized by exhaustion, cynicism, and reduced professional efficacy, and estimates of its prevalence vary significantly due to inconsistent measurement methods. Recent advancements in Natural Language Processing (NLP) and machine learning offer promising tools for detecting burnout through textual data analysis, with studies demonstrating high predictive accuracy. This paper contributes to burnout detection in German texts by: (a) collecting an anonymous real-world dataset including free-text answers and Oldenburg Burnout Inventory (OLBI) responses; (b) demonstrating the limitations of a GermanBERT-based classifier trained on online data; (c) presenting two versions of a curated BurnoutExpressions dataset, which yielded models that perform well in real-world applications; and (d) providing qualitative insights from an interdisciplinary focus group on the interpretability of AI models used for burnout detection. Our findings emphasize the need for greater collaboration between AI researchers and clinical experts to refine burnout detection models. Additionally, more real-world data is essential to validate and enhance the effectiveness of current AI methods developed in NLP research, which are often based on data automatically scraped from online sources and not evaluated in a real-world context. This is essential for ensuring AI tools are well suited for practical applications.
Unveiling Narrative Reasoning Limits of Large Language Models with Trope in Movie Synopses
Su, Hung-Ting, Hsu, Ya-Ching, Lin, Xudong, Shi, Xiang-Qian, Niu, Yulei, Hsu, Han-Yuan, Lee, Hung-yi, Hsu, Winston H.
Large language models (LLMs) equipped with chain-of-thoughts (CoT) prompting have shown significant multi-step reasoning capabilities in factual content like mathematics, commonsense, and logic. However, their performance in narrative reasoning, which demands greater abstraction capabilities, remains unexplored. This study utilizes tropes in movie synopses to assess the narrative reasoning abilities of state-of-the-art LLMs and uncovers their low performance. We introduce a trope-wise querying approach to address these challenges and boost the F1 score by 11.8 points. Moreover, Figure 1: While LLMs have revolutionized NLP reasoning, while prior studies suggest that CoT enhances surpassing previous supervised learning (SL) multi-step reasoning, this study shows methods and even reaching human-level performance CoT can cause hallucinations in narrative content, on some tasks, their limitations become apparent when reducing GPT-4's performance. We also tested against the Trope dataset. NLU: Natural Language introduce an Adversarial Injection method to Understanding, CS: Commonsense. Check Section embed trope-related text tokens into movie synopses 1 and 2.2 for details.
Some Mad Genius Put ChatGPT on a TI-84 Graphing Calculator
On Saturday, a YouTube creator called ChromaLock published a video detailing how he modified a Texas Instruments TI-84 graphing calculator to connect to the internet and access OpenAI's ChatGPT, potentially enabling students to cheat on tests. The video, titled "I Made the Ultimate Cheating Device," demonstrates a custom hardware modification that allows users of the graphing calculator to type in problems sent to ChatGPT using the keypad and receive live responses on the screen. ChromaLock began by exploring the calculator's link port, typically used for transferring educational programs between devices. He then designed a custom circuit board he calls "TI-32" that incorporates a tiny Wi-Fi-enabled microcontroller, the Seed Studio ESP32-C3 (which costs about 5), along with other components to interface with the calculator's systems. This story originally appeared on Ars Technica, a trusted source for technology news, tech policy analysis, reviews, and more.
OpenAI staffers reportedly 'taken aback' by 'ominous' logo rebranding
OpenAI could undergo massive changes next year, which include getting a brand new logo. According to Fortune, though, staff members were less than enthused when they got a sneak peek of its supposed new logo at a recent company-wide meeting. The company's hexagonal flower symbol, which has become pretty recognizable thanks to ChatGPT's popularity, is gone. Instead, it's replaced by a large black "O" or a simple ring or circle that staffers reportedly found to be devoid of creativity -- ominous, even. Based on how the publication's sources described it, the new logo sounds like the complete opposite of OpenAI's current one, which was designed to represent "precision, potential and optimism."
Meta Connect 2024: The cheaper Quest 3S, AI, smart glasses and everything else to expect
It used to go by at least two different names -- Oculus Connect and then Facebook Connect -- but whatever the moniker, Meta's fall event is still a big showcase for the company's latest and greatest achievements in the virtual reality and mixed reality space. Much like last year, we can likely predict the biggest news coming out of Meta Connect 2024 with just two acronyms: AI and AR. Like every other big tech firm this year, Meta will be desperate to demonstrate how it plans to stay relevant in a future powered by AI. And now that we're seven months beyond the launch of Apple's Vision Pro, which arrived alongside a short-lived spike in interest in augmented reality (AR), Meta CEO Mark Zuckerberg is likely eager to show off his own plans to make AR a reality. While Zuckerberg isn't as hot on the metaverse as he was when he renamed his company, the union of AI and AR is one way he can still make the dream of persistent virtual worlds come true.
How to get ChatGPT, Gemini, Midjourney, and more for life
TL;DR: Save 82% on a 1minAI lifetime subscription with ChatGPT, Gemini, and more platforms included. Everyone else is paying premium prices to get their favorite AI tools, but we'll let you in on a little secret: they're overpaying. We found this other tool that has ChatGPT, Gemini, Midjourney, and just about every other AI model you can think of built-in, except you don't have to pay any fees. Lifetime access to 1minAI is only 39.99 (reg. There's no need to have an account with any other AI provider.
StateAct: State Tracking and Reasoning for Acting and Planning with Large Language Models
Planning and acting to solve `real' tasks using large language models (LLMs) in interactive environments has become a new frontier for AI methods. While recent advances allowed LLMs to interact with online tools, solve robotics tasks and many more, long range reasoning tasks remain a problem for LLMs. Existing methods to address this issue are very resource intensive and require additional data or human crafted rules, instead, we propose a simple method based on few-shot in-context learning alone to enhance `chain-of-thought' with state-tracking for planning and acting with LLMs. We show that our method establishes the new state-of-the-art on Alfworld for in-context learning methods (\textbf{+14\%} over the previous best few-shot in-context learning method) and performs on par with methods that use additional training data and additional tools such as code-execution. We also demonstrate that our enhanced `chain-of-states' allows the agent to both solve longer horizon problems and to be more efficient in number of steps required to solve a task. We show that our method works across a variety of LLMs for both API-based and open source ones. Finally, we also conduct ablation studies and show that `chain-of-thoughts' helps state-tracking accuracy, while a json-structure harms overall performance. We open-source our code and annotations at \url{https://github.com/ai-nikolai/StateAct}.
A Survey on Multimodal Benchmarks: In the Era of Large AI Models
Li, Lin, Chen, Guikun, Shi, Hanrong, Xiao, Jun, Chen, Long
The rapid evolution of Multimodal Large Language Models (MLLMs) has brought substantial advancements in artificial intelligence, significantly enhancing the capability to understand and generate multimodal content. While prior studies have largely concentrated on model architectures and training methodologies, a thorough analysis of the benchmarks used for evaluating these models remains underexplored. This survey addresses this gap by systematically reviewing 211 benchmarks that assess MLLMs across four core domains: understanding, reasoning, generation, and application. We provide a detailed analysis of task designs, evaluation metrics, and dataset constructions, across diverse modalities. We hope that this survey will contribute to the ongoing advancement of MLLM research by offering a comprehensive overview of benchmarking practices and identifying promising directions for future work. An associated GitHub repository collecting the latest papers is available.
What is a Digital Twin Anyway? Deriving the Definition for the Built Environment from over 15,000 Scientific Publications
Abdelrahman, Mahmoud, Macatulad, Edgardo, Lei, Binyu, Quintana, Matias, Miller, Clayton, Biljecki, Filip
The concept of digital twins has attracted significant attention across various domains, particularly within the built environment. However, there is a sheer volume of definitions and the terminological consensus remains out of reach. The lack of a universally accepted definition leads to ambiguities in their conceptualization and implementation, and may cause miscommunication for both researchers and practitioners. We employed Natural Language Processing (NLP) techniques to systematically extract and analyze definitions of digital twins from a corpus of 15,000 full-text articles spanning diverse disciplines in the built environment. The study compares these findings with insights from an expert survey that included 52 experts. The study identifies concurrence on the components that comprise a 'Digital Twin' from a practical perspective across various domains, contrasting them with those that do not, to identify deviations. We investigate the evolution of digital twin definitions over time and across different scales, including manufacturing, building, and urban/geospatial perspectives. We extracted the main components of Digital Twins using Text Frequency Analysis and N-gram analysis. Subsequently, we identified components that appeared in the literature and conducted a Chi-square test to assess the significance of each component in different domains. Our findings indicate that definitions differ based on the field of research in which they are conceived, but with many similarities across domains. One significant generalizable differentiation is related to whether a digital twin was used for High-Performance Real-Time (HPRT) or Long-Term Decision Support (LTDS) applications. We synthesized and contrasted the most representative definitions in each domain, culminating in a novel, data-driven definition specifically tailored for each context.
LLM Agents as 6G Orchestrator: A Paradigm for Task-Oriented Physical-Layer Automation
Xiao, Zhuoran, Ye, Chenhui, Hu, Yunbo, Yuan, Honggang, Huang, Yihang, Feng, Yijia, Cai, Liyu, Chang, Jiang
The rapid advancement in generative pre-training models is propelling a paradigm shift in technological progression from basic applications such as chatbots towards more sophisticated agent-based systems. It is with huge potential and necessity that the 6G system be combined with the copilot of large language model (LLM) agents and digital twins (DT) to manage the highly complicated communication system with new emerging features such as native AI service and sensing. With the 6G-oriented agent, the base station could understand the transmission requirements of various dynamic upper-layer tasks, automatically orchestrate the optimal system workflow. Through continuously get feedback from the 6G DT for reinforcement, the agents can finally raise the performance of practical system accordingly. Differing from existing LLM agents designed for general application, the 6G-oriented agent aims to make highly rigorous and precise planning with a vast amount of extra expert knowledge, which inevitably requires a specific system design from model training to implementation. This paper proposes a novel comprehensive approach for building task-oriented 6G LLM agents. We first propose a two-stage continual pre-training and fine-tuning scheme to build the field basic model and diversities of specialized expert models for meeting the requirements of various application scenarios. Further, a novel inference framework based on semantic retrieval for leveraging the existing communication-related functions is proposed. Experiment results of exemplary tasks, such as physical-layer task decomposition, show the proposed paradigm's feasibility and effectiveness.