Data scarcity and class imbalance are persistent challenges in training robust NLP models, especially in specialized domains or low-resource settings. We propose a novel technique, SMOTExT, that adapts the idea of Synthetic Minority Over-sampling (SMOTE) to textual data. Our method generates new synthetic examples by interpolating between BERT-based embeddings of two existing examples and then decoding the resulting latent point into text with xRAG architecture. By leveraging xRAG's cross-modal retrieval-generation framework, we can effectively turn interpolated vectors into coherent text. While this is preliminary work supported by qualitative outputs only, the method shows strong potential for knowledge distillation and data augmentation in few-shot settings. Notably, our approach also shows promise for privacy-preserving machine learning: in early experiments, training models solely on generated data achieved comparable performance to models trained on the original dataset. This suggests a viable path toward safe and effective learning under data protection constraints.

I feel that both the precise training algorithm as well as the distinction of the two types of prototypes are important points to add to the revised version. I agree with the other reviewers that Section 3 can be written in a more clear way, and it would also be helpful to double-check the text for grammar for the final version. The problem of low-shot learning is to learn classifiers between a set of previously-unseen classes given only a few examples of each. It is assumed that a potentially large set of'training' or'base' classes is given, but this set is disjoint from the set of classes that will be presented at test time. Given the insufficiency of the training data for learning a low-shot classifier, generating additional data is a reasonable strategy and there has been increasingly more research on this recently (the authors correctly cite the relevant works).

We present WebGLM, a web-enhanced question-answering system based on the General Language Model (GLM). Its goal is to augment a pre-trained large language model (LLM) with web search and retrieval capabilities while being efficient for real-world deployments. To achieve this, we develop WebGLM with strategies for the LLM-augmented retriever, bootstrapped generator, and human preference-aware scorer. Specifically, we identify and address the limitations of WebGPT (OpenAI), through which WebGLM is enabled with accuracy, efficiency, and cost-effectiveness advantages. In addition, we propose systematic criteria for evaluating web-enhanced QA systems. We conduct multi-dimensional human evaluation and quantitative ablation studies, which suggest the outperformance of the proposed WebGLM designs over existing systems. WebGLM with the 10-billion-parameter GLM (10B) is shown to perform better than the similar-sized WebGPT (13B) and even comparably to WebGPT (175B) in human evaluation. The code, demo, and data are at \url{https://github.com/THUDM/WebGLM}.

Domain Generalization (DG) is an important open problem in machine learning. Deep models are susceptible to domain shifts of even minute degrees, which severely compromises their reliability in real applications. To alleviate the issue, most existing methods enforce various invariant constraints across multiple training domains. However,such an approach provides little performance guarantee for novel test domains in general. In this paper, we investigate a different approach named Contrastive Domain Generalization (CDG), which exploits semantic invariance exhibited by strongly contrastive data pairs in lieu of multiple domains. We present a causal DG theory that shows the potential capability of CDG; together with a regularization technique, Logit Attribution Matching (LAM), for realizing CDG. We empirically show that LAM outperforms state-of-the-art DG methods with only a small portion of paired data and that LAM helps models better focus on semantic features which are crucial to DG.

With the rise of low power speech-enabled devices, there is a growing demand to quickly produce models for recognizing arbitrary sets of keywords. As with many machine learning tasks, one of the most challenging parts in the model creation process is obtaining a sufficient amount of training data. In this paper, we explore the effectiveness of synthesized speech data in training small, spoken term detection models of around 400k parameters. Instead of training such models directly on the audio or low level features such as MFCCs, we use a pre-trained speech embedding model trained to extract useful features for keyword spotting models. Using this speech embedding, we show that a model which detects 10 keywords when trained on only synthetic speech is equivalent to a model trained on over 500 real examples. We also show that a model without our speech embeddings would need to be trained on over 4000 real examples to reach the same accuracy.

Perhaps you've read about AI capable of producing humanlike speech or generating images of people that are difficult to distinguish from real-life photographs. More often than not, these systems build upon generative adversarial networks (GANs), which are two-part AI models consisting of a generator that creates samples and a discriminator that attempts to differentiate between the generated samples and real-world samples. This unique arrangement enables GANs to achieve impressive feats of media synthesis, from composing melodies and swapping sheep for giraffes to hallucinating footage of ice skaters and soccer players. In point of fact, it's because of this prowess that GANs have been used to produce problematic content like deepfakes, which is media that takes a person in existing media and replaces them with someone else's likeness. The evolution of GANs -- which Facebook AI research director Yann LeCun has called the most interesting idea of the decade -- is somewhat long and winding, and very much continues to this day.

Machine learning is an ever-evolving field, so it can be easy to feel like you're out of the loop on the latest developments changing the world this week. One of those emerging areas that have been getting a lot of buzz lately is GANs--or generative adversarial networks. So to keep you in the machine learning loop, we've put together a short crash course on GANs: With generative models, the aim is to model the distribution of a given dataset. For the generative models that we're talking about today, that dataset is usually a set of images, but it could also be other kinds of data, like audio samples or time-series data. There are two ways to go about getting a model of this distribution: implicitly or explicitly.

Sometimes it is not enough for a DNN to produce an outcome. For example, in applications such as healthcare, users need to understand the rationale of the decisions. Therefore, it is imperative to develop algorithms to learn models with good interpretability (Doshi-Velez 2017). An important factor that leads to the lack of interpretability of DNNs is the ambiguity of neurons, where a neuron may fire for various unrelated concepts. This work aims to increase the interpretability of DNNs on the whole image space by reducing the ambiguity of neurons. In this paper, we make the following contributions: 1) We propose a metric to evaluate the consistency level of neurons in a network quantitatively. 2) We find that the learned features of neurons are ambiguous by leveraging adversarial examples. 3) We propose to improve the consistency of neurons on adversarial example subset by an adversarial training algorithm with a consistent loss.

When you buy an item on a favored website, does the site show you pictures of what others have bought? Retailers have been building such systems for years, many built using the programming language R. For older implementations of recommender systems, it's time to consider improving performance and scalability by moving these systems to the cloud --the Azure cloud. Recently, we were asked to help a customer improve the performance and process surrounding the R implementation of their recommender solution and host the model in Azure. Many of their early analytic products were built in R, and they wanted to preserve that investment. After a review of their solution, we identified bottlenecks that could be vanquished.

A new expert report on AI sentiment among the public finds that the industry has a lot of work to do to explain the technology's benefits – not with marketing hype, but with real-world examples and proof. Consumers lack understanding about artificial intelligence (AI) and are looking to businesses, government, and academia for hard facts, according to a new report. This lack of insight into the technology is fuelling fears about its potential impact, the report found, while the industry isn't helping its own cause by making bold claims about AI without providing verifiable evidence. The Future, surveyed consumers across the US and UK about their current sentiments on AI. The corporate communications company also sought the views of a panel of 25 global experts in the fields of AI and robotics.