Near out-of-distribution detection (OOD) is a major challenge for deep neural networks. We demonstrate that large-scale pre-trained transformers can significantly improve the state-of-the-art (SOTA) on a range of near OOD tasks across different data modalities. For instance, on CIFAR-100 vs CIFAR-10 OOD detection, we improve the AUROC from 85% (current SOTA) to 96% using Vision Transformers pre-trained on ImageNet-21k. On a challenging genomics OOD detection benchmark, we improve the AUROC from 66% to 77% using transformers and unsupervised pre-training. To further improve performance, we explore the few-shot outlier exposure setting where a few examples from outlier classes may be available; we show that pre-trained transformers are particularly well-suited for outlier exposure, and that the AUROC of OOD detection on CIFAR-100 vs CIFAR10 can be improved to 98.7% with just 1 image per OOD class, and 99.46% with 10 images per OOD class. For multi-modal image-text pre-trained transformers such as CLIP, we explore a new way of using just the names of outlier classes as a sole source of information without any accompanying images, and show that this outperforms previous SOTA on standard vision OOD benchmark tasks.

Modeling customer shopping intentions is a crucial task for e-commerce, as it directly impacts user experience and engagement. Thus, accurately understanding customer preferences is essential for providing personalized recommendations. Session-based recommendation, which utilizes customer session data to predict their next interaction, has become increasingly popular. However, existing session datasets have limitations in terms of item attributes, user diversity, and dataset scale. As a result, they cannot comprehensively capture the spectrum of user behaviors and preferences.

Ace can read the trajectory of a ball, adjust the racket angle, and respond with strokes that keep the exchange alive with real players. Ace has won three out of five games played under official rules. Ace is a robot that aims high: It wants to become the world champion of table tennis . It was developed by Sony AI researchers who, in a new study published in Nature, have shown how this robot, equipped with artificial intelligence, has faced some high-level athletes, holding its own in matches played according to the official rules of table tennis. This feat represents a milestone for the world of robotics, a field that has long regarded this sport, among the most technical in the world, as one of the most difficult tests of technological advances.

Large Transformer models have been central to recent advances in natural language processing. The training and inference costs of these models, however, have grown rapidly and become prohibitively expensive. Here we aim to reduce the costs of Transformers by searching for a more efficient variant. Compared to previous approaches, our search is performed at a lower level, over the primitives that define a Transformer TensorFlow program. We identify an architecture, named Primer, that has a smaller training cost than the original Transformer and other variants for auto-regressive language modeling.

When you purchase through links in our articles, we may earn a small commission. Why choose one AI model when this $79.97 Lock in lifetime access to 1minAI's Advanced Business Plan for $79.97 (MSRP $540) through April 26 -- and tap into multiple top AI models from one dashboard. There's a point where testing prompts across ChatGPT, Gemini, and everything else stops feeling productive and starts feeling like busywork. It's built to be approachable but still powerful, giving you flexibility without the usual friction.

When solving decision-making tasks, humans typically depend on information from two key sources: (1) Historical policy data, which provides interaction replay from the environment, and (2) Analytical insights in natural language form, exposing the invaluable thought process or strategic considerations. Despite this, the majority of preceding research focuses on only one source: they either use historical replay exclusively to directly learn policy or value functions, or engaged in language model training utilizing mere language corpus. In this paper, we argue that a powerful autonomous agent should cover both sources. Thus, we propose ChessGPT, a GPT model bridging policy learning and language modeling by integrating data from these two sources in Chess games. Specifically, we build a large-scale game and language dataset related to chess.

In addition to CYCLIP described in 2, we train two more instantiations of it by keeping either of the two consistency regularizers active in the loss objective (Eq. The instantiation trained by setting λ1 = 0and λ2 = 0.5is termed as C-CYCLIP as only cross-modal consistency regularizer term is added to the loss objective. Similarly, we get I-CYCLIP where only in-modal consistency regularizer is added to the loss by setting λ1 = 0.5 and λ2 = 0. We evaluate C-CYCLIP and I-CYCLIP on most of the experiments discussed in the main text to understand their zero-shot transfer ability on standard datasets and robustness to natural distribution shifts. A.1 Zero-shot Transfer Table 7 presents our results of the zero-shot transfer experiment described in 3.1. We find that CYCLIP outperforms its sub-variants and the CLIP model on the ImageNet1K dataset.