Silicon Valley is not a milieu known for glamour and charisma. Still, Peter Thiel has cultivated a mystique. A billionaire several times over, Thiel was the first outside investor in Facebook; he went on to co-found PayPal, the digital-payment service, and Palantir, the data-intelligence company that has worked with the U.S. government. He has co-written a business best-seller, "Zero to One," and launched a hedge fund; he now runs three venture-capital firms. In 2018, citing a regional intolerance of conservative perspectives, he moved from Silicon Valley to Los Angeles; he recently purchased a mansion in Miami Beach.

The predominant approach for language modeling is to process sequences from left to right, but this eliminates a source of information: the order by which the sequence was generated. One strategy to recover this information is to decode both the content and ordering of tokens. Existing approaches supervise content and ordering by designing problem-specific loss functions and pre-training with an ordering pre-selected. Other recent works use iterative search to discover problem-specific orderings for training, but suffer from high time complexity and cannot be efficiently parallelized. We address these limitations with an unsupervised parallelizable learner that discovers high-quality generation orders purely from training data -- no domain knowledge required. The learner contains an encoder network and decoder language model that perform variational inference with autoregressive orders (represented as permutation matrices) as latent variables. The corresponding ELBO is not differentiable, so we develop a practical algorithm for end-to-end optimization using policy gradients. We implement the encoder as a Transformer with non-causal attention that outputs permutations in one forward pass. Permutations then serve as target generation orders for training an insertion-based Transformer language model. Empirical results in language modeling tasks demonstrate that our method is context-aware and discovers orderings that are competitive with or even better than fixed orders.

State-of-the-art approaches to reasoning and question answering over knowledge graphs (KGs) usually scale with the number of edges and can only be applied effectively on small instance-dependent subgraphs. In this paper, we address this issue by showing that multi-hop and more complex logical reasoning can be accomplished separately without losing expressive power. Motivated by this insight, we propose an approach to multi-hop reasoning that scales linearly with the number of relation types in the graph, which is usually significantly smaller than the number of edges or nodes. This produces a set of candidate solutions that can be provably refined to recover the solution to the original problem. Our experiments on knowledge-based question answering show that our approach solves the multi-hop MetaQA dataset, achieves a new state-of-the-art on the more challenging WebQuestionsSP, is orders of magnitude more scalable than competitive approaches, and can achieve compositional generalization out of the training distribution.

Knowledge graphs (KGs) have become widespread, and various knowledge graphs are constructed incessantly to support many in-KG and out-of-KG applications. During the construction of KGs, although new KGs may contain new entities with respect to constructed KGs, some entity-independent knowledge can be transferred from constructed KGs to new KGs. We call such knowledge meta-knowledge, and refer to the problem of transferring meta-knowledge from constructed (source) KGs to new (target) KGs to improve the performance of tasks on target KGs as meta-knowledge transfer for knowledge graphs. However, there is no available general framework that can tackle meta-knowledge transfer for both in-KG and out-of-KG tasks uniformly. Therefore, in this paper, we propose a framework, MorsE, which means conducting Meta-Learning for Meta-Knowledge Transfer via Knowledge Graph Embedding. MorsE represents the meta-knowledge via Knowledge Graph Embedding and learns the meta-knowledge by Meta-Learning. Specifically, MorsE uses an entity initializer and a Graph Neural Network (GNN) modulator to entity-independently obtain entity embeddings given a KG and is trained following the meta-learning setting to gain the ability of effectively obtaining embeddings. Experimental results on meta-knowledge transfer for both in-KG and out-of-KG tasks show that MorsE is able to learn and transfer meta-knowledge between KGs effectively, and outperforms existing state-of-the-art models.

Pactera EDGE, a world-class digital solutions provider for the data-driven, intelligent enterprise, announced the appointment of Vasudevan Sundarababu as a Senior Vice President, Head of Digital Engineering. Sundarababu, who has over 25-years of IT industry experience, most recently served as Global Head of Cloud Data Platforms for Capgemini Financial Services. He was previously Chief Technology Officer of CSS Corp. In his new role, Sundarababu will lead Pactera EDGE's global digital engineering practice, where he will be responsible for the identification and design of new products and solutions, the development of technology strategies and capabilities, and the inception of programs to bring these opportunities to Pactera EDGE's clients. Additionally, he will provide support to the sales team for client proposals and solutions.

Expert.ai announced that its natural language (NL) API providing deep language understanding is now available in the AWS Marketplace, a digital catalog with thousands of software listings from independent software vendors that make it easy to find, test, buy, and deploy software that runs on Amazon Web Services (AWS). NL API is a powerful way to structure unstructured language data leveraging deep language intelligence with minimal effort. The API identifies which meaning of a word is used in context ("disambiguation") to quickly analyze text for key elements, relations, classifications and more. It can also determine sentiment and even capture a range of 117 behavioral and emotional traits, providing the richest, most comprehensive and granular emotional and behavioral taxonomy available throughout the AI-based API ecosystem. Furthermore, using built-in technologies and its extensive knowledge graph, the expert.ai NL API can be used in more targeted ways to identify sensitive data (to protect customers, victims, users or research subjects, as well as to comply with data privacy regulations), media-related topics, geographical taxonomies and more.

In 1901, on an excavation trip to Crete, British archaeologist Arthur Evans unearthed items he believed belonged to a royal game dating back millennia: a board fashioned out of ivory, gold, silver, and rock crystals, and four conical pieces nearby, assumed to be the tokens. Playing it, however, stumped Evans, and many others after him who took a stab at it. There was no rulebook, no hints, and no other copies have ever been found. Games need instructions for players to follow. Without any, the Greek board's function remained unresolved--that is, until recently. Enter artificial intelligence, and a group of researchers from Maastricht University in the Netherlands.

Researchers have mapped the web of connections underpinning coronavirus conspiracy theories, opening a new way of understanding and challenging them. Using Danish witchcraft folklore as a model, the researchers from UCLA and Berkeley analysed thousands of social media posts with an artificial intelligence tool and extracted the key people, things and relationships. The tool enabled them to piece together the underlying stories in coronavirus conspiracy theories from fragments in online posts. One discovery from the research identifies Bill Gates as the reason why conspiracy theorists connect 5G with the virus. With Gates' background in computer technology and vaccination programmes, he served as a shortcut for these storytellers to link the two.

We propose defensive tensorization, an adversarial defence technique that leverages a latent high-order factorization of the network. The layers of a network are first expressed as factorized tensor layers. Tensor dropout is then applied in the latent subspace, therefore resulting in dense reconstructed weights, without the sparsity or perturbations typically induced by the randomization.Our approach can be readily integrated with any arbitrary neural architecture and combined with techniques like adversarial training. We empirically demonstrate the effectiveness of our approach on standard image classification benchmarks. We validate the versatility of our approach across domains and low-precision architectures by considering an audio classification task and binary networks. In all cases, we demonstrate improved performance compared to prior works.

Recent studies show that private training data can be leaked through the gradients sharing mechanism deployed in distributed machine learning systems, such as federated learning (FL). Increasing batch size to complicate data recovery is often viewed as a promising defense strategy against data leakage. In this paper, we revisit this defense premise and propose an advanced data leakage attack with theoretical justification to efficiently recover batch data from the shared aggregated gradients. We name our proposed method as catastrophic data leakage in vertical federated learning (CAFE). Comparing to existing data leakage attacks, our extensive experimental results on vertical FL settings demonstrate the effectiveness of CAFE to perform large-batch data leakage attack with improved data recovery quality. We also propose a practical countermeasure to mitigate CAFE. Our results suggest that private data participated in standard FL, especially the vertical case, have a high risk of being leaked from the training gradients. Our analysis implies unprecedented and practical data leakage risks in those learning settings. The code of our work is available at https://github.com/DeRafael/CAFE.