Srihari, Rohini K.
Beyond Discrete Personas: Personality Modeling Through Journal Intensive Conversations
Pal, Sayantan, Das, Souvik, Srihari, Rohini K.
Large Language Models (LLMs) have significantly improved personalized conversational capabilities. However, existing datasets like Persona Chat, Synthetic Persona Chat, and Blended Skill Talk rely on static, predefined personas. This approach often results in dialogues that fail to capture human personalities' fluid and evolving nature. To overcome these limitations, we introduce a novel dataset with around 400,000 dialogues and a framework for generating personalized conversations using long-form journal entries from Reddit. Our approach clusters journal entries for each author and filters them by selecting the most representative cluster, ensuring that the retained entries best reflect the author's personality. We further refine the data by capturing the Big Five personality traits --openness, conscientiousness, extraversion, agreeableness, and neuroticism --ensuring that dialogues authentically reflect an individual's personality. Using Llama 3 70B, we generate high-quality, personality-rich dialogues grounded in these journal entries. Fine-tuning models on this dataset leads to an 11% improvement in capturing personality traits on average, outperforming existing approaches in generating more coherent and personality-driven dialogues.
Mitigating Clickbait: An Approach to Spoiler Generation Using Multitask Learning
Pal, Sayantan, Das, Souvik, Srihari, Rohini K.
This study introduces 'clickbait spoiling', a novel technique designed to detect, categorize, and generate spoilers as succinct text responses, countering the curiosity induced by clickbait content. By leveraging a multi-task learning framework, our model's generalization capabilities are significantly enhanced, effectively addressing the pervasive issue of clickbait. The crux of our research lies in generating appropriate spoilers, be it a phrase, an extended passage, or multiple, depending on the spoiler type required. Our methodology integrates two crucial techniques: a refined spoiler categorization method and a modified version of the Question Answering (QA) mechanism, incorporated within a multi-task learning paradigm for optimized spoiler extraction from context. Notably, we have included fine-tuning methods for models capable of handling longer sequences to accommodate the generation of extended spoilers. This research highlights the potential of sophisticated text processing techniques in tackling the omnipresent issue of clickbait, promising an enhanced user experience in the digital realm.
Improving Dialog Safety using Socially Aware Contrastive Learning
Das, Souvik, Srihari, Rohini K.
State-of-the-art conversational AI systems raise concerns due to their potential risks of generating unsafe, toxic, unethical, or dangerous content. Previous works have developed datasets to teach conversational agents the appropriate social paradigms to respond effectively to specifically designed hazardous content. However, models trained on these adversarial datasets still struggle to recognize subtle unsafe situations that appear naturally in conversations or introduce an inappropriate response in a casual context. To understand the extent of this problem, we study prosociality in both adversarial and casual dialog contexts and audit the response quality of general-purpose language models in terms of propensity to produce unsafe content. We propose a dual-step fine-tuning process to address these issues using a socially aware n-pair contrastive loss. Subsequently, we train a base model that integrates prosocial behavior by leveraging datasets like Moral Integrity Corpus (MIC) and ProsocialDialog. Experimental results on several dialog datasets demonstrate the effectiveness of our approach in generating socially appropriate responses.
Domain Specific Question Answering Over Knowledge Graphs Using Logical Programming and Large Language Models
Madani, Navid, Srihari, Rohini K., Joseph, Kenneth
Question Answering over Knowledge Graphs We propose an approach that utilizes LLMs to represent (KGQA) poses significant challenges in the field questions within a specific domain, extracting of Natural Language Processing (NLP). As structured their meanings, while employing logical programming knowledge graphs capturing rich semantic techniques for reasoning and knowledge information become prevalent, there is a pressing representation. Our objective is to demonstrate need for intelligent systems that can reason effectively how this integration enables robust and adaptable and provide accurate answers to intricate KGQA systems that can navigate domain-specific questions within specific domains. The primary knowledge graphs and provide accurate answers to focus of KGQA is to bridge the gap between human complex questions. To evaluate the effectiveness language and structured knowledge representations. of our proposed approach, we conduct experiments When presented with a question in natural using the MetaQA dataset (Zhang et al., 2018), language, KGQA systems aim to traverse the a widely adopted benchmark in KGQA research.
Diving Deep into Modes of Fact Hallucinations in Dialogue Systems
Das, Souvik, Saha, Sougata, Srihari, Rohini K.
Knowledge Graph(KG) grounded conversations often use large pre-trained models and usually suffer from fact hallucination. Frequently entities with no references in knowledge sources and conversation history are introduced into responses, thus hindering the flow of the conversation -- existing work attempt to overcome this issue by tweaking the training procedure or using a multi-step refining method. However, minimal effort is put into constructing an entity-level hallucination detection system, which would provide fine-grained signals that control fallacious content while generating responses. As a first step to address this issue, we dive deep to identify various modes of hallucination in KG-grounded chatbots through human feedback analysis. Secondly, we propose a series of perturbation strategies to create a synthetic dataset named FADE (FActual Dialogue Hallucination DEtection Dataset). Finally, we conduct comprehensive data analyses and create multiple baseline models for hallucination detection to compare against human-verified data and already established benchmarks.
Assessing Effectiveness of Using Internal Signals for Check-Worthy Claim Identification in Unlabeled Data for Automated Fact-Checking
Pathak, Archita, Srihari, Rohini K.
While recent work on automated fact-checking has focused mainly on verifying and explaining claims, for which the list of claims is readily available, identifying check-worthy claim sentences from a text remains challenging. Current claim identification models rely on manual annotations for each sentence in the text, which is an expensive task and challenging to conduct on a frequent basis across multiple domains. This paper explores methodology to identify check-worthy claim sentences from fake news articles, irrespective of domain, without explicit sentence-level annotations. We leverage two internal supervisory signals - headline and the abstractive summary - to rank the sentences based on semantic similarity. We hypothesize that this ranking directly correlates to the check-worthiness of the sentences. To assess the effectiveness of this hypothesis, we build pipelines that leverage the ranking of sentences based on either the headline or the abstractive summary. The top-ranked sentences are used for the downstream fact-checking tasks of evidence retrieval and the article's veracity prediction by the pipeline. Our findings suggest that the top 3 ranked sentences contain enough information for evidence-based fact-checking of a fake news article. We also show that while the headline has more gisting similarity with how a fact-checking website writes a claim, the summary-based pipeline is the most promising for an end-to-end fact-checking system.
The 1995 Fall Symposia Series
Cohn, David, Lewis, David, Aha, David W., Burke, Robin, Srihari, Rohini K., Horswill, Ian, Buvac, Sasa, Siegel, Eric V., Fehling, Michael
The Association for the Advancement of Artificial Intelligence (AAAI) held its 1995 Fall Symposia Series on 10 to 12 November in Cambridge, Massachusetts. This article contains summaries of the eight symposia that were conducted: (1) Active Learning; (2) Adaptation of Knowledge for Reuse; (3) AI Applications in Knowledge Navigation and Retrieval; (4) Computational Models for Integrating Language and Vision; (5) Embodied Language and Action Symposium; (6) Formalizing Context; (7) Genetic Programming; and (8) Rational Agency: Concepts, Theories, Models, and Applications.
The 1995 Fall Symposia Series
Cohn, David, Lewis, David, Aha, David W., Burke, Robin, Srihari, Rohini K., Horswill, Ian, Buvac, Sasa, Siegel, Eric V., Fehling, Michael
The Association for the Advancement of Artificial Intelligence (AAAI) held its 1995 Fall Symposia Series on 10 to 12 November in Cambridge, Massachusetts. This article contains summaries of the eight symposia that were conducted: (1) Active Learning; (2) Adaptation of Knowledge for Reuse; (3) AI Applications in Knowledge Navigation and Retrieval; (4) Computational Models for Integrating Language and Vision; (5) Embodied Language and Action Symposium; (6) Formalizing Context; (7) Genetic Programming; and (8) Rational Agency: Concepts, Theories, Models, and Applications.