Large Language Model
Retention Is All You Need
Mohiuddin, Karishma, Alam, Mirza Ariful, Alam, Mirza Mohtashim, Welke, Pascal, Martin, Michael, Lehmann, Jens, Vahdati, Sahar
Skilled employees are the most important pillars of an organization. Despite this, most organizations face high attrition and turnover rates. While several machine learning models have been developed to analyze attrition and its causal factors, the interpretations of those models remain opaque. In this paper, we propose the HR-DSS approach, which stands for Human Resource (HR) Decision Support System, and uses explainable AI for employee attrition problems. The system is designed to assist HR departments in interpreting the predictions provided by machine learning models. In our experiments, we employ eight machine learning models to provide predictions. We further process the results achieved by the best-performing model by the SHAP explainability process and use the SHAP values to generate natural language explanations which can be valuable for HR. Furthermore, using "What-if-analysis", we aim to observe plausible causes for attrition of an individual employee. The results show that by adjusting the specific dominant features of each individual, employee attrition can turn into employee retention through informative business decisions.
Protecting Language Generation Models via Invisible Watermarking
Zhao, Xuandong, Wang, Yu-Xiang, Li, Lei
Language generation models have been an increasingly powerful enabler for many applications. Many such models offer free or affordable API access, which makes them potentially vulnerable to model extraction attacks through distillation. To protect intellectual property (IP) and ensure fair use of these models, various techniques such as lexical watermarking and synonym replacement have been proposed. However, these methods can be nullified by obvious countermeasures such as "synonym randomization". To address this issue, we propose GINSEW, a novel method to protect text generation models from being stolen through distillation. The key idea of our method is to inject secret signals into the probability vector of the decoding steps for each target token. We can then detect the secret message by probing a suspect model to tell if it is distilled from the protected one. Experimental results show that GINSEW can effectively identify instances of IP infringement with minimal impact on the generation quality of protected APIs. Our method demonstrates an absolute improvement of 19 to 29 points on mean average precision (mAP) in detecting suspects compared to previous methods against watermark removal attacks.
The Heated Debate Over Who Should Control Access to AI
In May, the CEOs of three of the most prominent AI labs--OpenAI, Google DeepMind, and Anthropic--signed a statement that warned AI could be as risky to humanity as pandemics and nuclear war. To prevent disaster, many AI companies and researchers are arguing for restrictions on who can access the most powerful AI models and who can develop them in the first place. They worry that bad actors could use AI models to create large amounts of disinformation that could alter the outcomes of elections, and that in the future, more powerful AI models could help launch cyberattacks or create bioweapons. But not all AI companies agree. On Thursday, Meta released Code Llama, a family of AI models built on top of Llama 2, Meta's flagship large language model, with extra training to make them particularly useful for coding tasks.
AI Can't Read Books. It's Reviewing Them Anyway
Now that we've all had experience with large language models, their limitations are all too visible. But their prose doesn't explode in the mind like the words of Jennifer Egan, Emily St. John Mandel, or David Foster Wallace do. Yes they can make music. But Taylor Swift and Kendrick Lamar are sleeping very well at night. And they sure can summarize history speedily and neatly, but not with the perspicacity of Barbara Tuchman or Ron Chernow.
Chris Christie calls out Vivek Ramaswamy for GOP primary debate performance: Uses 'ChatGPT phrases'
Former New Jersey Gov. Chris Christie tore into GOP presidential candidate Vivek Ramaswamy one day after the first primary debate in Milwaukee, arguing the entrepreneur's answers showed he has "absolutely no idea what he's talking about." Christie and Ramaswamy sparred over several issues during the two-hour debate from the United States' role in funding the war in Ukraine to supporting former President Donald Trump if he's convicted. Trump praised Ramaswamy's debate performance on his social media site Truth Social. Ramaswamy also praised Trump on stage as the "best president of the 21st century." "Well, I'm stunned that as I was talking about Donald Trump and all the ways that he's let down our party and our country, that he [Trump] didn't mention me as a winner of the debate last night," Christie said Thursday on "Your World."
New York Times, CNN and ABC block OpenAI's GPTBot web crawler from accessing content
News outlets including the New York Times, CNN, Reuters and the Australian Broadcasting Corporation (ABC) have blocked a tool from OpenAI, limiting the company's ability to continue accessing their content. OpenAI is behind one of the best known artificial intelligence chatbots, ChatGPT. Its web crawler – known as GPTBot – may scan webpages to help improve its AI models. The Verge was first to report the New York Times had blocked GPTBot on its website. The Guardian subsequently found that other major news websites, including CNN, Reuters, the Chicago Tribune, the ABC and Australian Community Media (ACM) brands such as the Canberra Times and the Newcastle Herald, appear to have also disallowed the web crawler.
1.5 million materials narratives generated by chatbots
Park, Yang Jeong, Jerng, Sung Eun, Park, Jin-Sung, Kwon, Choah, Hsu, Chia-Wei, Ren, Zhichu, Yoon, Sungroh, Li, Ju
The advent of artificial intelligence (AI) has enabled a comprehensive exploration of materials for various applications. However, AI models often prioritize frequently encountered materials in the scientific literature, limiting the selection of suitable candidates based on inherent physical and chemical properties. To address this imbalance, we have generated a dataset of 1,494,017 natural language-material paragraphs based on combined OQMD, Materials Project, JARVIS, COD and AFLOW2 databases, which are dominated by ab initio calculations and tend to be much more evenly distributed on the periodic table. The generated text narratives were then polled and scored by both human experts and ChatGPT-4, based on three rubrics: technical accuracy, language and structure, and relevance and depth of content, showing similar scores but with human-scored depth of content being the most lagging. The merger of multi-modality data sources and large language model (LLM) holds immense potential for AI frameworks to help the exploration and discovery of solid-state materials for specific applications.
Rethinking Language Models as Symbolic Knowledge Graphs
Mruthyunjaya, Vishwas, Pezeshkpour, Pouya, Hruschka, Estevam, Bhutani, Nikita
Symbolic knowledge graphs (KGs) play a pivotal role in knowledge-centric applications such as search, question answering and recommendation. As contemporary language models (LMs) trained on extensive textual data have gained prominence, researchers have extensively explored whether the parametric knowledge within these models can match up to that present in knowledge graphs. Various methodologies have indicated that enhancing the size of the model or the volume of training data enhances its capacity to retrieve symbolic knowledge, often with minimal or no human supervision. Despite these advancements, there is a void in comprehensively evaluating whether LMs can encompass the intricate topological and semantic attributes of KGs, attributes crucial for reasoning processes. In this work, we provide an exhaustive evaluation of language models of varying sizes and capabilities. We construct nine qualitative benchmarks that encompass a spectrum of attributes including symmetry, asymmetry, hierarchy, bidirectionality, compositionality, paths, entity-centricity, bias and ambiguity. Additionally, we propose novel evaluation metrics tailored for each of these attributes. Our extensive evaluation of various LMs shows that while these models exhibit considerable potential in recalling factual information, their ability to capture intricate topological and semantic traits of KGs remains significantly constrained. We note that our proposed evaluation metrics are more reliable in evaluating these abilities than the existing metrics. Lastly, some of our benchmarks challenge the common notion that larger LMs (e.g., GPT-4) universally outshine their smaller counterparts (e.g., BERT).
Integrating LLMs and Decision Transformers for Language Grounded Generative Quality-Diversity
Salehi, Achkan, Doncieux, Stephane
Quality-Diversity is a branch of stochastic optimization that is often applied to problems from the Reinforcement Learning and control domains in order to construct repertoires of well-performing policies/skills that exhibit diversity with respect to a behavior space. Such archives are usually composed of a finite number of reactive agents which are each associated to a unique behavior descriptor, and instantiating behavior descriptors outside of that coarsely discretized space is not straight-forward. While a few recent works suggest solutions to that issue, the trajectory that is generated is not easily customizable beyond the specification of a target behavior descriptor. We propose to jointly solve those problems in environments where semantic information about static scene elements is available by leveraging a Large Language Model to augment the repertoire with natural language descriptions of trajectories, and training a policy conditioned on those descriptions. Thus, our method allows a user to not only specify an arbitrary target behavior descriptor, but also provide the model with a high-level textual prompt to shape the generated trajectory. We also propose an LLM-based approach to evaluating the performance of such generative agents. Furthermore, we develop a benchmark based on simulated robot navigation in a 2d maze that we use for experimental validation.
LLM2KB: Constructing Knowledge Bases using instruction tuned context aware Large Language Models
Nayak, Anmol, Timmapathini, Hari Prasad
The advent of Large Language Models (LLM) has revolutionized the field of natural language processing, enabling significant progress in various applications. One key area of interest is the construction of Knowledge Bases (KB) using these powerful models. Knowledge bases serve as repositories of structured information, facilitating information retrieval and inference tasks. Our paper proposes LLM2KB, a system for constructing knowledge bases using large language models, with a focus on the Llama 2 architecture and the Wikipedia dataset. We perform parameter efficient instruction tuning for Llama-2-13b-chat and StableBeluga-13B by training small injection models that have only 0.05 % of the parameters of the base models using the Low-Rank Adaptation (LoRA) technique. These injection models have been trained with prompts that are engineered to utilize Wikipedia page contexts of subject entities fetched using a Dense Passage Retrieval (DPR) algorithm, to answer relevant object entities for a given subject entity and relation. Our best performing model achieved an average F1 score of 0.6185 across 21 relations in the LM-KBC challenge held at the ISWC 2023 conference.