In this work, we address the problem of directing the text generation of a language model (LM) towards a desired behavior, aligning the generated text with the preferences of the human operator. We propose using another, instruction-tuned language model as a critic reward model in a zero-shot way thanks to the prompt of a Yes-No question that represents the user preferences, without requiring further labeled data. This zero-shot reward model provides the learning signal to further fine-tune the base LM using Reinforcement Learning from AI Feedback (RLAIF); yet our approach is also compatible in other contexts such as quality-diversity search. Extensive evidence of the capabilities of the proposed ZYN framework is provided through experiments in different domains related to text generation, including detoxification; optimizing sentiment of movie reviews, or any other attribute; steering the opinion about a particular topic the model may have; and personalizing prompt generators for text-to-image tasks. Code available at \url{https://github.com/vicgalle/zero-shot-reward-models/}.

The remarkable performance of pre-trained large language models has revolutionised various natural language processing applications. Due to huge parametersizes and extensive running costs, companies or organisations tend to transfer the models to the target task by zero-shot prompting techniques. However, the prohibitive costs of tokens and time have hindered their adoption in applications. We propose OverPrompt, leveraging the in-context learning capability of LLMs to handle multiple task inputs, thereby reducing token and time costs. This approach could potentially improve task performance during API queries due to better conditional distribution mapping. Evaluated across diverse classification datasets, our experiments show that OverPrompt can achieve cost-efficient zero-shot classification without causing significant detriment to task performance, and in some cases, even improving it. An ablation study conducted on various LLMs, along with an investigation into the robustness of our prompting strategy to different input ordering, offers valuable insights into the broader applicability of our method across diverse tasks. These findings also suggest a more seamless integration of our method with LLMs through an API.

Large language models (LLMs) have shown remarkable performance in various tasks and have been extensively utilized by the public. However, the increasing concerns regarding the misuse of LLMs, such as plagiarism and spamming, have led to the development of multiple detectors, including fine-tuned classifiers and statistical methods. In this study, we equip LLMs with prompts, rather than relying on an external paraphraser, to evaluate the vulnerability of these detectors. We propose a novel Substitution-based In-Context example Optimization method (SICO) to automatically construct prompts for evading the detectors. SICO is cost-efficient as it requires only 40 human-written examples and a limited number of LLM inferences to generate a prompt. Moreover, once a task-specific prompt has been constructed, it can be universally used against a wide range of detectors. Extensive experiments across three real-world tasks demonstrate that SICO significantly outperforms the paraphraser baselines and enables GPT-3.5 to successfully evade six detectors, decreasing their AUC by 0.5 on average. Furthermore, a comprehensive human evaluation as well as a validation experiment in the wild show that the SICO-generated text achieves human-level readability and task completion rates. Finally, the strong performance of SICO exhibits its potential as a reliable evaluation tool for future detectors.

The first step to apply deep learning techniques for symbolic music understanding is to transform musical pieces (mainly in MIDI format) into sequences of predefined tokens like note pitch, note velocity, and chords. Subsequently, the sequences are fed into a neural sequence model to accomplish specific tasks. Music sequences exhibit strong correlations between adjacent elements, making them prime candidates for N-gram techniques from Natural Language Processing (NLP). Consider classical piano music: specific melodies might recur throughout a piece, with subtle variations each time. In this paper, we propose a novel method, NG-Midiformer, for understanding symbolic music sequences that leverages the N-gram approach. Our method involves first processing music pieces into word-like sequences with our proposed unsupervised compoundation, followed by using our N-gram Transformer encoder, which can effectively incorporate N-gram information to enhance the primary encoder part for better understanding of music sequences. The pre-training process on large-scale music datasets enables the model to thoroughly learn the N-gram information contained within music sequences, and subsequently apply this information for making inferences during the fine-tuning stage. Experiment on various datasets demonstrate the effectiveness of our method and achieved state-of-the-art performance on a series of music understanding downstream tasks. The code and model weights will be released at https://github.com/CinqueOrigin/NG-Midiformer.

We extend our previous work on Inductive Conformal Prediction (ICP) for multi-label text classification and present a novel approach for addressing the computational inefficiency of the Label Powerset (LP) ICP, arrising when dealing with a high number of unique labels. We present experimental results using the original and the proposed efficient LP-ICP on two English and one Czech language data-sets. Specifically, we apply the LP-ICP on three deep Artificial Neural Network (ANN) classifiers of two types: one based on contextualised (bert) and two on non-contextualised (word2vec) word-embeddings. In the LP-ICP setting we assign nonconformity scores to label-sets from which the corresponding p-values and prediction-sets are determined. Our approach deals with the increased computational burden of LP by eliminating from consideration a significant number of label-sets that will surely have p-values below the specified significance level. This reduces dramatically the computational complexity of the approach while fully respecting the standard CP guarantees. Our experimental results show that the contextualised-based classifier surpasses the non-contextualised-based ones and obtains state-of-the-art performance for all data-sets examined. The good performance of the underlying classifiers is carried on to their ICP counterparts without any significant accuracy loss, but with the added benefits of ICP, i.e. the confidence information encapsulated in the prediction sets. We experimentally demonstrate that the resulting prediction sets can be tight enough to be practically useful even though the set of all possible label-sets contains more than $1e+16$ combinations. Additionally, the empirical error rates of the obtained prediction-sets confirm that our outputs are well-calibrated.

There is a tension at the heart of ChatGPT that may soon snap. Does the technology expand our world or constrain it? Which is to say, do AI-powered chatbots open new doors to learning and discovery, or do they instead risk siloing off information and leaving us stuck with unreliable access to truth? Earlier today, OpenAI, the maker of ChatGPT, announced a partnership with the media conglomerate Axel Springer that seems to get us closer to an answer. Under the arrangement, ChatGPT will gain the capacity to present its users with "summaries of selected global news content" published by the news organizations in Axel Springer's portfolio, which includes Politico and Business Insider.

Axel Springer, the publisher of Business Insider and Politico, said on Wednesday it was partnering with OpenAI, which will pay the German media group to allow ChatGPT to summarize current articles in responses generated by the chatbot. "ChatGPT users around the world will receive summaries of selected global news content from Axel Springer's media brands," which also includes the German tabloid Bild, the two companies said in a statement. The chatbot's answers will include material otherwise kept behind a paywall and offer "links to the full articles for transparency and further information", they said. Axel Springer will be paid for making its content available to the US artificial intelligence firm, a spokesman for the media group told AFP. The deal is valid for several years and does not commit either side to exclusivity, leaving them free to sign new agreements, the spokesman said without giving more detail.

The Electronic Entertainment Expo, perhaps the most over-the-top, bombastic event ever to be designated an industry trade show, is no more. E3 was a staple of the video game calendar for over two decades, showing off the latest and greatest in gaming hardware and software every summer. But it's been officially declared dead by the ESA, in the wake of diminishing trade shows worldwide post-Covid pandemic. E3's demise wasn't exactly shocking, since it hasn't held a live, in-person event since 2019. But the closure of such a high-profile event has some people wondering: Is CES, the electronics industry's most high-profile event, next on the chopping block?

OpenAI will pay German publisher Axel Springer to use its news articles to train its AI models and show real-time information from Axel Springer's brands, which include Business Insider and Politico in the US and Bild and Welt in Europe, in ChatGPT's responses. None of the companies disclosed how much the deal was worth, but Bloomberg reported that OpenAI will pay the publisher tens of millions of euros over the next three years. "This partnership with Axel Springer will help provide people with new ways to access quality, real-time news content through our AI tools," said OpenAI's chief operating officer Brad Lightcap in a statement. "We are deeply committed to working with publishers and creators around the world and ensuring they benefit from advanced AI technology and new revenue models." OpenAI's partnership with Axel Springer comes on the heels of concerns from creators, authors, and publishers who have criticized and sued generative AI companies for training their models on their content without consent or compensation.

This week, the panel is joined by Slate writer and senior editor Sam Adams to dissect Poor Things, director Yorgos Lanthimos horny, steampunk Frankenstein tale about Bella Baxter (played by Emma Stone), a pregnant woman who commits suicide then is brought back to life by a brilliant scientist (Willem Dafoe), with an eccentric caveat: She now has the brain of her unborn fetus. Then, the three remember Norman Lear, the late television pioneer and American icon who died at the age of 101 and who was responsible for ushering in a new era of character-driven, comedic, topical, and morally serious TV with hit sitcoms like All in the Family, The Jeffersons, Maude, and One Day at a Time. Finally, they are joined by Slate's books and culture columnist, Laura Miller, who shares her top ten books of the year, and along with Dana, discusses the joys and challenges of year-end listmaking. In the exclusive Slate Plus segment, the panel reunites with Sam Adams to spoil Poor Things, detailing what is arguably the film's weakest portion: the final ten minutes. The deadline to submit is Wednesday, December 13.