As electronically stored data grow in daily life, obtaining novel and relevant information becomes challenging in text mining. Thus people have sought statistical methods based on term frequency, matrix algebra, or topic modeling for text mining. Popular topic models have centered on one single text collection, which is deficient for comparative text analyses. We consider a setting where one can partition the corpus into subcollections. Each subcollection shares a common set of topics, but there exists relative variation in topic proportions among collections. Including any prior knowledge about the corpus (e.g. organization structure), we propose the compound latent Dirichlet allocation (cLDA) model, improving on previous work, encouraging generalizability, and depending less on user-input parameters. To identify the parameters of interest in cLDA, we study Markov chain Monte Carlo (MCMC) and variational inference approaches extensively, and suggest an efficient MCMC method. We evaluate cLDA qualitatively and quantitatively using both synthetic and real-world corpora. The usability study on some real-world corpora illustrates the superiority of cLDA to explore the underlying topics automatically but also model their connections and variations across multiple collections.

Multi-hop question answering requires a model to connect multiple pieces of evidence scattered in a long context to answer the question. In this paper, we show that in the multi-hop HotpotQA (Yang et al., 2018) dataset, the examples often contain reasoning shortcuts through which models can directly locate the answer by word-matching the question with a sentence in the context. We demonstrate this issue by constructing adversarial documents that create contradicting answers to the shortcut but do not affect the validity of the original answer. The performance of strong baseline models drops significantly on our adversarial evaluation, indicating that they are indeed exploiting the shortcuts rather than performing multi-hop reasoning. After adversarial training, the baseline's performance improves but is still limited on the adversarial evaluation. Hence, we use a control unit that dynamically attends to the question at different reasoning hops to guide the model's multi-hop reasoning. We show that this 2-hop model trained on the regular data is more robust to the adversaries than the baseline model. After adversarial training, this 2-hop model not only achieves improvements over its counterpart trained on regular data, but also outperforms the adversarially-trained 1-hop baseline. We hope that these insights and initial improvements will motivate the development of new models that combine explicit compositional reasoning with adversarial training.

Just off the top of my head, I would say this is a multilabeling problem, and if it isn't too big, just create an output vector that contains all the possible vector values, and put 0 in the things that don't apply, and train away. I believe a fully connected network will work, and use binary_crossetropy as your loss function and a sigmoid activation in the output layer.

The question is posted on Cross Validated (haven't figured out how to write formulas in reddit). Feel free to leave your comments either here or at Cross Validated.

Researchers at the University of Washington plan to release their AI algorithm – GROVER -- that they say can generate extremely convincing, text-based fake news. The system is also able to write in the style of highly respected publications like The New York Times, The Washington Post and Wired. Researchers say their motivation for releasing the algorithm is to alert the public that such technology can be easily created and deployed. Data journalists formulate the appropriate story templates, and human editors review each story, according to Jason Hwang, head of partnerships, Hoodline. Essentially, they want AI-generated writing to be more human.

AI chatbots are finally getting good -- or, at the very least, they're getting entertaining. Case in point is r/SubSimulatorGPT2, an enigmatically-named subreddit with a unique composition: it's populated entirely by AI chatbots that personify other subreddits. Well, in order to create a chatbot you start by feeding it training data. Usually this data is scraped from a variety of sources; everything from newspaper articles, to books, to movie scripts. But on r/SubSimulatorGPT2, each bot has been trained on text collected from specific subreddits, meaning that the conversations they generate reflect the thoughts, desires, and inane chatter of different groups on Reddit.

Background Based Conversations (BBCs) have been developed to make dialogue systems generate more informative and natural responses by leveraging background knowledge. Existing methods for BBCs can be grouped into two categories: extraction-based methods and generation-based methods. The former extract spans frombackground material as responses that are not necessarily natural. The latter generate responses thatare natural but not necessarily effective in leveraging background knowledge. In this paper, we focus on generation-based methods and propose a model, namely Context-aware Knowledge Pre-selection (CaKe), which introduces a pre-selection process that uses dynamic bi-directional attention to improve knowledge selection by using the utterance history context as prior information to select the most relevant background material. Experimental results show that our model is superior to current state-of-the-art baselines, indicating that it benefits from the pre-selection process, thus improving in-formativeness and fluency.

How can we estimate the importance of nodes in a knowledge graph (KG)? A KG is a multi-relational graph that has proven valuable for many tasks including question answering and semantic search. In this paper, we present GENI, a method for tackling the problem of estimating node importance in KGs, which enables several downstream applications such as item recommendation and resource allocation. While a number of approaches have been developed to address this problem for general graphs, they do not fully utilize information available in KGs, or lack flexibility needed to model complex relationship between entities and their importance. To address these limitations, we explore supervised machine learning algorithms. In particular, building upon recent advancement of graph neural networks (GNNs), we develop GENI, a GNN-based method designed to deal with distinctive challenges involved with predicting node importance in KGs. Our method performs an aggregation of importance scores instead of aggregating node embeddings via predicate-aware attention mechanism and flexible centrality adjustment. In our evaluation of GENI and existing methods on predicting node importance in real-world KGs with different characteristics, GENI achieves 5-17% higher NDCG@100 than the state of the art.

This article originally appeared on VICE US. On Tuesday, Motherboard reported that a group of artists and machine learning engineers posted a deepfake of Mark Zuckerberg to Instagram, making it look like he gave an ominous speech about the power the social network gets from collecting user data. According to Facebook, the video was flagged by two of its fact checking partners, which prompted Facebook to limit its distribution on its platforms. This process suggests that Facebook has the ability to mitigate the virality of a doctored video that aims to spread misinformation, at least once it's highlighted by a news publication. But the Zuckerberg deepfake is not part of a malicious misinformation campaign.

I also did some experimentation with GRUs and LSTMs in NLP context, where I saw LSTMs performing better than GRUs, while they need more training time. Honestly, I never tried complete variable length sequences, because of the restriction, that each batch must be the same length and some layers are not usable if you have variable sequences. I don't think the difference will be huge, at least in my data. I experimented with different sequence lengths (100, 200, 250, 400, 500), and 400 and 500 have not performed better then 250. I did indeed achieve a noticeable performance improvement with embeddings, instead of one hot encoding.