We combine the capacity of sparsely gated Mixture-of-Experts (MoE) with the speed and stability of linear, mixing transformations to design the Sparse Mixer encoder model. Sparse Mixer slightly outperforms (<1%) BERT on GLUE and SuperGLUE, but more importantly trains 65% faster and runs inference 61% faster. We also present a faster variant, prosaically named Fast Sparse Mixer, that marginally underperforms BERT on SuperGLUE, but trains and runs nearly twice as fast. We justify the design of these two models by carefully ablating through various mixing mechanisms, MoE configurations and hyperparameters. Sparse Mixer overcomes many of the latency and stability concerns of MoE models and offers the prospect of serving sparse student models, without resorting to distilling them to dense variants.

Context-aware recommendation systems improve upon classical recommender systems by including, in the modelling, a user's behaviour. Research into context-aware recommendation systems has previously only considered the sequential ordering of items as contextual information. However, there is a wealth of unexploited additional multi-modal information available in auxiliary knowledge related to items. This study extends the existing research by evaluating a multi-modal recommendation system that exploits the inclusion of comprehensive auxiliary knowledge related to an item. The empirical results explore extracting vector representations (embeddings) from unstructured and structured data using data2vec. The fused embeddings are then used to train several state-of-the-art transformer architectures for sequential user-item representations. The analysis of the experimental results shows a statistically significant improvement in prediction accuracy, which confirms the effectiveness of including auxiliary information in a context-aware recommendation system. We report a 4% and 11% increase in the NDCG score for long and short user sequence datasets, respectively.

The graph represents a network of 1,277 Twitter users whose tweets in the requested range contained "#selfdrivingcars", or who were replied to or mentioned in those tweets. The network was obtained from the NodeXL Graph Server on Wednesday, 12 October 2022 at 12:40 UTC. The requested start date was Wednesday, 12 October 2022 at 00:01 UTC and the maximum number of tweets (going backward in time) was 7,500. The tweets in the network were tweeted over the 10-day, 14-hour, 54-minute period from Saturday, 01 October 2022 at 08:17 UTC to Tuesday, 11 October 2022 at 23:12 UTC. Additional tweets that were mentioned in this data set were also collected from prior time periods.

GITEX Global–Uniphore, the leader in Conversational AI and Automation, at GITEX Global 2022, announced a strategic partnership with Avaya, a global leader in solutions to enhance and simplify communications and collaboration, to bring its integrated Conversational AI and communications platform to customers across the Middle East and African (MEA) region. "In today's uncertain world, consumers want brands to address their needs quickly and efficiently; this makes the customer experience more important than ever" Uniphore's Conversational AI and Automation products will add deep functionality to the Avaya OneCloud CCaaS platform. Avaya OneCloud CCaaS makes it easy to connect chat, video, voice, and messaging to deliver enhanced experiences for customers and employees at every touchpoint. With Uniphore, Avaya OneCloud CCaaS users will be able to track, measure, and improve their contact center journey with increased self-serve capabilities, frictionless agent experience, and needle-moving insights. Avaya's customers will have access to Uniphore's conversational AI and automation solutions and will be well-placed to digitally onboard customers, including from social media platforms driven by AI-powered solutions.

We hear from Steven Kolawole about his paper on sign-to-speech models for Nigerian sign language. Steven told us about the goals of this research, his methodology, and how the work has inspired research in other languages. The biggest goal of the research was to reduce the communication barrier between the hearing-impaired community and the general populace, focusing on sub-Saharan Africa. Sub-Saharan Africa is one of the regions with the highest number of cases of hearing disabilities and, additionally, the region with the lowest number of solutions targeted towards solving this problem. And investigating why this is the status quo was very interesting.

Ai-Da the robot has admitted she was'nervous' about speaking at the House of Lords and named her favourite artist as Yoko Ono in an exclusive interview with MailOnline. Ai-Da made history on Tuesday by becoming the first robot to address the House of Lords – although she suffered a slight hiccup after'falling asleep' mid-speech. During the session, the bot had to be rebooted by her creator Aidan Meller, after a technical issue rendered her cross-eyed and zombie-like. Shortly after, MailOnline asked Ai-Da a couple of questions about the address. Wearing dungarees and an orange blouse, Ai-Da said the address to the House of Lords went well and that she feels'quite nervous when speaking in public' Ai-Da is an artificial intelligence robot built in 2019 that creates drawings, paintings and sculptures.

AlphaTensor was designed to perform matrix multiplications, but the same approach could be used to tackle other mathematical challenges.Credit: DeepMind An artificial intelligence (AI) developed by machine-learning company DeepMind in London has tackled a type of calculation called matrix multiplication. The system -- called AlphaTensor -- leverages the skills that DeepMind's game-playing AIs use to beat human players at games such as Go and chess. Matrix multiplication is a widely used mathematical technique that involves multiplying numbers arranged in grids, or matrices, that might represent sets of pixels in images, air conditions in a weather model or the internal workings of an artificial neural network. AlphaTensor broke ground by finding shortcuts to solve these problems with fewer steps (A. The same general approach could have applications in other kinds of mathematical operation, its developers say, such as decomposing complex waves or other mathematical objects into simpler ones.

A good state representation is crucial to solving complicated reinforcement learning (RL) challenges. Many recent works focus on designing auxiliary losses for learning informative representations. Unfortunately, these handcrafted objectives rely heavily on expert knowledge and may be sub-optimal. In this paper, we propose a principled and universal method for learning better representations with auxiliary loss functions, named Automated Auxiliary Loss Search (A2LS), which automatically searches for top-performing auxiliary loss functions for RL. Specifically, based on the collected trajectory data, we define a general auxiliary loss space of size $7.5 \times 10^{20}$ and explore the space with an efficient evolutionary search strategy. Empirical results show that the discovered auxiliary loss (namely, A2-winner) significantly improves the performance on both high-dimensional (image) and low-dimensional (vector) unseen tasks with much higher efficiency, showing promising generalization ability to different settings and even different benchmark domains. We conduct a statistical analysis to reveal the relations between patterns of auxiliary losses and RL performance.

Leveraging task-aware annotated data as supervised signals to assist with self-supervised learning on large-scale unlabeled data has become a new trend in pre-training language models. Existing studies show that multi-task learning with large-scale supervised tasks suffers from negative effects across tasks. To tackle the challenge, we propose a task prefix guided multi-task pre-training framework to explore the relationships among tasks. We conduct extensive experiments on 40 datasets, which show that our model can not only serve as the strong foundation backbone for a wide range of tasks but also be feasible as a probing tool for analyzing task relationships. The task relationships reflected by the prefixes align transfer learning performance between tasks. They also suggest directions for data augmentation with complementary tasks, which help our model achieve human-parity results on commonsense reasoning leaderboards. Code is available at https://github.com/cooelf/CompassMTL

Revision game is a very new model formulating the real-time situation where players dynamically prepare and revise their actions in advance before a deadline when payoffs are realized. It is at the cutting edge of dynamic game theory and can be applied in many real-world scenarios, such as eBay auction, stock market, election, online games, crowdsourcing, etc. In this work, we novelly identify a class of strategies for revision games which are called Limited Retaliation strategies. An limited retaliation strategy stipulates that, (1) players first follow a recommended cooperative plan; (2) if anyone deviates from the plan, the limited retaliation player retaliates by using the defection action for a limited duration; (3) after the retaliation, the limited retaliation player returns to the cooperative plan. A limited retaliation strategy has three key features. It is cooperative, sustaining a high level of social welfare. It is vengeful, deterring the opponent from betrayal by threatening with a future retaliation. It is yet forgiving, since it resumes cooperation after a proper retaliation. The cooperativeness and vengefulness make it constitute cooperative subgame perfect equilibrium, while the forgiveness makes it tolerate occasional mistakes. limited retaliation strategies show significant advantages over Grim Trigger, which is currently the only known strategy for revision games. Besides its contribution as a new robust and welfare-optimizing equilibrium strategy, our results about limited retaliation strategy can also be used to explain how easy cooperation can happen, and why forgiveness emerges in real-world multi-agent interactions. In addition, limited retaliation strategies are simple to derive and computationally efficient, making it easy for algorithm design and implementation in many multi-agent systems.