Personal Assistant Systems
Intention Knowledge Graph Construction for User Intention Relation Modeling
Bai, Jiaxin, Wang, Zhaobo, Cheng, Junfei, Yu, Dan, Huang, Zerui, Wang, Weiqi, Liu, Xin, Luo, Chen, He, Qi, Zhu, Yanming, Li, Bo, Song, Yangqiu
Understanding user intentions is challenging for online platforms. Recent work on intention knowledge graphs addresses this but often lacks focus on connecting intentions, which is crucial for modeling user behavior and predicting future actions. This paper introduces a framework to automatically generate an intention knowledge graph, capturing connections between user intentions. Using the Amazon m2 dataset, we construct an intention graph with 351 million edges, demonstrating high plausibility and acceptance. Our model effectively predicts new session intentions and enhances product recommendations, outperforming previous state-of-the-art methods and showcasing the approach's practical utility.
Matrix Completion via Residual Spectral Matching
Noisy matrix completion has attracted significant attention due to its applications in recommendation systems, signal processing and image restoration. Most existing works rely on (weighted) least squares methods under various low-rank constraints. However, minimizing the sum of squared residuals is not always efficient, as it may ignore the potential structural information in the residuals. In this study, we propose a novel residual spectral matching criterion that incorporates not only the numerical but also locational information of residuals. This criterion is the first in noisy matrix completion to adopt the perspective of low-rank perturbation of random matrices and exploit the spectral properties of sparse random matrices. We derive optimal statistical properties by analyzing the spectral properties of sparse random matrices and bounding the effects of low-rank perturbations and partial observations. Additionally, we propose algorithms that efficiently approximate solutions by constructing easily computable pseudo-gradients. The iterative process of the proposed algorithms ensures convergence at a rate consistent with the optimal statistical error bound. Our method and algorithms demonstrate improved numerical performance in both simulated and real data examples, particularly in environments with high noise levels.
A multi-theoretical kernel-based approach to social network-based recommendation
Li, Xin, Wang, Mengyue, Liang, T. -P.
Recommender systems are a critical component of e-commercewebsites. The rapid development of online social networking services provides an opportunity to explore social networks together with information used in traditional recommender systems, such as customer demographics, product characteristics, and transactions. It also provides more applications for recommender systems. To tackle this social network-based recommendation problem, previous studies generally built trust models in light of the social influence theory. This study inspects a spectrumof social network theories to systematicallymodel themultiple facets of a social network and infer user preferences. In order to effectively make use of these heterogonous theories, we take a kernel-based machine learning paradigm, design and select kernels describing individual similarities according to social network theories, and employ a non-linear multiple kernel learning algorithm to combine the kernels into a unified model. This design also enables us to consider multiple theories' interactions in assessing individual behaviors. We evaluate our proposed approach on a real-world movie review data set. The experiments show that our approach provides more accurate recommendations than trust-based methods and the collaborative filtering approach. Further analysis shows that kernels derived from contagion theory and homophily theory contribute a larger portion of the model.
RecSys Arena: Pair-wise Recommender System Evaluation with Large Language Models
Wu, Zhuo, Jia, Qinglin, Wu, Chuhan, Du, Zhaocheng, Wang, Shuai, Wang, Zan, Dong, Zhenhua
Evaluating the quality of recommender systems is critical for algorithm design and optimization. Most evaluation methods are computed based on offline metrics for quick algorithm evolution, since online experiments are usually risky and time-consuming. However, offline evaluation usually cannot fully reflect users' preference for the outcome of different recommendation algorithms, and the results may not be consistent with online A/B test. Moreover, many offline metrics such as AUC do not offer sufficient information for comparing the subtle differences between two competitive recommender systems in different aspects, which may lead to substantial performance differences in long-term online serving. Fortunately, due to the strong commonsense knowledge and role-play capability of large language models (LLMs), it is possible to obtain simulated user feedback on offline recommendation results. Motivated by the idea of LLM Chatbot Arena, in this paper we present the idea of RecSys Arena, where the recommendation results given by two different recommender systems in each session are evaluated by an LLM judger to obtain fine-grained evaluation feedback. More specifically, for each sample we use LLM to generate a user profile description based on user behavior history or off-the-shelf profile features, which is used to guide LLM to play the role of this user and evaluate the relative preference for two recommendation results generated by different models. Through extensive experiments on two recommendation datasets in different scenarios, we demonstrate that many different LLMs not only provide general evaluation results that are highly consistent with canonical offline metrics, but also provide rich insight in many subjective aspects. Moreover, it can better distinguish different algorithms with comparable performance in terms of AUC and nDCG.
Modeling the Heterogeneous Duration of User Interest in Time-Dependent Recommendation: A Hidden Semi-Markov Approach
Zhang, Haidong, Ni, Wancheng, Li, Xin, Yang, Yiping
Recommender systems are widely used for suggesting books, education materials, and products to users by exploring their behaviors. In reality, users' preferences often change over time, leading to studies on time-dependent recommender systems. However, most existing approaches that deal with time information remain primitive. In this paper, we extend existing methods and propose a hidden semi-Markov model to track the change of users' interests. Particularly, this model allows for capturing the different durations of user stays in a (latent) interest state, which can better model the heterogeneity of user interests and focuses. We derive an expectation maximization algorithm to estimate the parameters of the framework and predict users' actions. Experiments on three real-world datasets show that our model significantly outperforms the state-of-the-art time-dependent and static benchmark methods. Further analyses of the experiment results indicate that the performance improvement is related to the heterogeneity of state durations and the drift of user interests in the dataset.
Enhancing Discoverability in Enterprise Conversational Systems with Proactive Question Suggestions
Shen, Xiaobin, Lee, Daniel, Ranjan, Sumit, Harsha, Sai Sree, Sevak, Pawan, Li, Yunyao
Enterprise conversational AI systems are becoming increasingly popular to assist users in completing daily tasks such as those in marketing and customer management. However, new users often struggle to ask effective questions, especially in emerging systems with unfamiliar or evolving capabilities. This paper proposes a framework to enhance question suggestions in conversational enterprise AI systems by generating proactive, context-aware questions that try to address immediate user needs while improving feature discoverability. Our approach combines periodic user intent analysis at the population level with chat session-based question generation. We evaluate the framework using real-world data from the AI Assistant for Adobe Experience Platform (AEP), demonstrating the improved usefulness and system discoverability of the AI Assistant.
DK-PRACTICE: An Intelligent Educational Platform for Personalized Learning Content Recommendations Based on Students Knowledge State
Delianidi, Marina, Diamantaras, Konstantinos, Moras, Ioannis, Sidiropoulos, Antonis
This study introduces DK-PRACTICE (Dynamic Knowledge Prediction and Educational Content Recommendation System), an intelligent online platform that leverages machine learning to provide personalized learning recommendations based on student knowledge state. Students participate in a short, adaptive assessment using the question-and-answer method regarding key concepts in a specific knowledge domain. The system dynamically selects the next question for each student based on the correctness and accuracy of their previous answers. After the test is completed, DK-PRACTICE analyzes students' interaction history to recommend learning materials to empower the student's knowledge state in identified knowledge gaps. Both question selection and learning material recommendations are based on machine learning models trained using anonymized data from a real learning environment. To provide self-assessment and monitor learning progress, DK-PRACTICE allows students to take two tests: one pre-teaching and one post-teaching. After each test, a report is generated with detailed results. In addition, the platform offers functions to visualize learning progress based on recorded test statistics. DK-PRACTICE promotes adaptive and personalized learning by empowering students with self-assessment capabilities and providing instructors with valuable information about students' knowledge levels. DK-PRACTICE can be extended to various educational environments and knowledge domains, provided the necessary data is available according to the educational topics. A subsequent paper will present the methodology for the experimental application and evaluation of the platform.
Whom do Explanations Serve? A Systematic Literature Survey of User Characteristics in Explainable Recommender Systems Evaluation
Wardatzky, Kathrin, Inel, Oana, Rossetto, Luca, Bernstein, Abraham
Adding explanations to recommender systems is said to have multiple benefits, such as increasing user trust or system transparency. Previous work from other application areas suggests that specific user characteristics impact the users' perception of the explanation. However, we rarely find this type of evaluation for recommender systems explanations. This paper addresses this gap by surveying 124 papers in which recommender systems explanations were evaluated in user studies. We analyzed their participant descriptions and study results where the impact of user characteristics on the explanation effects was measured. Our findings suggest that the results from the surveyed studies predominantly cover specific users who do not necessarily represent the users of recommender systems in the evaluation domain. This may seriously hamper the generalizability of any insights we may gain from current studies on explanations in recommender systems. We further find inconsistencies in the data reporting, which impacts the reproducibility of the reported results. Hence, we recommend actions to move toward a more inclusive and reproducible evaluation.
From Lived Experience to Insight: Unpacking the Psychological Risks of Using AI Conversational Agents
Chandra, Mohit, Naik, Suchismita, Ford, Denae, Okoli, Ebele, De Choudhury, Munmun, Ershadi, Mahsa, Ramos, Gonzalo, Hernandez, Javier, Bhattacharjee, Ananya, Warreth, Shahed, Suh, Jina
Recent gain in popularity of AI conversational agents has led to their increased use for improving productivity and supporting well-being. While previous research has aimed to understand the risks associated with interactions with AI conversational agents, these studies often fall short in capturing the lived experiences. Additionally, psychological risks have often been presented as a sub-category within broader AI-related risks in past taxonomy works, leading to under-representation of the impact of psychological risks of AI use. To address these challenges, our work presents a novel risk taxonomy focusing on psychological risks of using AI gathered through lived experience of individuals. We employed a mixed-method approach, involving a comprehensive survey with 283 individuals with lived mental health experience and workshops involving lived experience experts to develop a psychological risk taxonomy. Our taxonomy features 19 AI behaviors, 21 negative psychological impacts, and 15 contexts related to individuals. Additionally, we propose a novel multi-path vignette based framework for understanding the complex interplay between AI behaviors, psychological impacts, and individual user contexts. Finally, based on the feedback obtained from the workshop sessions, we present design recommendations for developing safer and more robust AI agents. Our work offers an in-depth understanding of the psychological risks associated with AI conversational agents and provides actionable recommendations for policymakers, researchers, and developers.
Opinion de-polarization of social networks with GNNs
Mylonas, Konstantinos, Spyropoulos, Thrasyvoulos
Nowadays, social media is the ground for political debate and exchange of opinions. There is a significant amount of research that suggests that social media are highly polarized. A phenomenon that is commonly observed is the echo chamber structure, where users are organized in polarized communities and form connections only with similar-minded individuals, limiting themselves to consume specific content. In this paper we explore a way to decrease the polarization of networks with two echo chambers. Particularly, we observe that if some users adopt a moderate opinion about a topic, the polarization of the network decreases. Based on this observation, we propose an efficient algorithm to identify a good set of K users, such that if they adopt a moderate stance around a topic, the polarization is minimized. Our algorithm employs a Graph Neural Network and thus it can handle large graphs more effectively than other approaches