Large language models (LLMs) have shown remarkable performance in various tasks and have been extensively utilized by the public. However, the increasing concerns regarding the misuse of LLMs, such as plagiarism and spamming, have led to the development of multiple detectors, including fine-tuned classifiers and statistical methods. In this study, we equip LLMs with prompts, rather than relying on an external paraphraser, to evaluate the vulnerability of these detectors. We propose a novel Substitution-based In-Context example Optimization method (SICO) to automatically construct prompts for evading the detectors. SICO is cost-efficient as it requires only 40 human-written examples and a limited number of LLM inferences to generate a prompt. Moreover, once a task-specific prompt has been constructed, it can be universally used against a wide range of detectors. Extensive experiments across three real-world tasks demonstrate that SICO significantly outperforms the paraphraser baselines and enables GPT-3.5 to successfully evade six detectors, decreasing their AUC by 0.5 on average. Furthermore, a comprehensive human evaluation as well as a validation experiment in the wild show that the SICO-generated text achieves human-level readability and task completion rates. Finally, the strong performance of SICO exhibits its potential as a reliable evaluation tool for future detectors.

Foundation models are trained on increasingly immense and opaque datasets. Even while these models are now key in AI system building, it can be difficult to answer the straightforward question: has the model already encountered a given example during training? We therefore propose a widespread adoption of Data Portraits: artifacts that record training data and allow for downstream inspection. First we outline the properties of such an artifact and discuss how existing solutions can be used to increase transparency. We then propose and implement a solution based on data sketching, stressing fast and space efficient querying. Using our tools, we document a popular language modeling corpus (The Pile) and a recently released code modeling dataset (The Stack). We show that our solution enables answering questions about test set leakage and model plagiarism. Our tool is lightweight and fast, costing only 3% of the dataset size in overhead. We release a live interface of our tools at https://dataportraits.org/ and call on dataset and model creators to release Data Portraits as a complement to current documentation practices.

Humanoid robots that can autonomously operate in diverse environments have the potential to help address labour shortages in factories, assist elderly at homes, and colonize new planets. While classical controllers for humanoid robots have shown impressive results in a number of settings, they are challenging to generalize and adapt to new environments. Here, we present a fully learning-based approach for real-world humanoid locomotion. Our controller is a causal transformer that takes the history of proprioceptive observations and actions as input and predicts the next action. We hypothesize that the observation-action history contains useful information about the world that a powerful transformer model can use to adapt its behavior in-context, without updating its weights. We train our model with large-scale model-free reinforcement learning on an ensemble of randomized environments in simulation and deploy it to the real world zero-shot. Our controller can walk over various outdoor terrains, is robust to external disturbances, and can adapt in context.

Citation generation aims to generate a citation sentence that refers to a chosen paper in the context of a manuscript. However, a rigid citation generation process is at odds with an author's desire to control specific attributes, such as 1) the citation intent, e.g., either introducing background information or comparing results, and 2) keywords that should appear in the citation text. To provide these degrees of controllability during citation generation, we propose to integrate the manuscript context, the context of the referenced paper, and the desired control attributes into a structured template and use it to fine-tune a language model (LM) via next-token prediction. We then utilize Proximal Policy Optimization to directly optimize the LM in favor of a high score of our proposed controllability metric. The proposed workflow harmoniously combines citation attribute suggestion and conditional citation generation into one LM, allowing for better user control.

We release and introduce the TigerBot family of large language models (LLMs), consisting of base and chat models, sized from 7, 13, 70 and 180 billion parameters. We develop our models embarking from Llama-2 and BLOOM, and push the boundary further in data, training algorithm, infrastructure, and application tools. Our models yield meaningful performance gain over SOTA open-source models, e.g., Llama-2, specifically 6% gain in English and 20% gain in Chinese. TigerBot model family also achieves leading performance in major academic and industrial benchmarks and leaderboards. We believe that TigerBot represents just a snapshot of lightning-fast progression in LLM open-source community. Therefore, we are thrilled to give back by publicly releasing our models and reporting our approach behind, with additional emphases on building SOTA LLMs in a democratized way and making LLMs of use in real-world applications.

There is a tension at the heart of ChatGPT that may soon snap. Does the technology expand our world or constrain it? Which is to say, do AI-powered chatbots open new doors to learning and discovery, or do they instead risk siloing off information and leaving us stuck with unreliable access to truth? Earlier today, OpenAI, the maker of ChatGPT, announced a partnership with the media conglomerate Axel Springer that seems to get us closer to an answer. Under the arrangement, ChatGPT will gain the capacity to present its users with "summaries of selected global news content" published by the news organizations in Axel Springer's portfolio, which includes Politico and Business Insider.

Axel Springer, the publisher of Business Insider and Politico, said on Wednesday it was partnering with OpenAI, which will pay the German media group to allow ChatGPT to summarize current articles in responses generated by the chatbot. "ChatGPT users around the world will receive summaries of selected global news content from Axel Springer's media brands," which also includes the German tabloid Bild, the two companies said in a statement. The chatbot's answers will include material otherwise kept behind a paywall and offer "links to the full articles for transparency and further information", they said. Axel Springer will be paid for making its content available to the US artificial intelligence firm, a spokesman for the media group told AFP. The deal is valid for several years and does not commit either side to exclusivity, leaving them free to sign new agreements, the spokesman said without giving more detail.

OpenAI will pay German publisher Axel Springer to use its news articles to train its AI models and show real-time information from Axel Springer's brands, which include Business Insider and Politico in the US and Bild and Welt in Europe, in ChatGPT's responses. None of the companies disclosed how much the deal was worth, but Bloomberg reported that OpenAI will pay the publisher tens of millions of euros over the next three years. "This partnership with Axel Springer will help provide people with new ways to access quality, real-time news content through our AI tools," said OpenAI's chief operating officer Brad Lightcap in a statement. "We are deeply committed to working with publishers and creators around the world and ensuring they benefit from advanced AI technology and new revenue models." OpenAI's partnership with Axel Springer comes on the heels of concerns from creators, authors, and publishers who have criticized and sued generative AI companies for training their models on their content without consent or compensation.

WHEN New Scientist editors sat down to discuss the biggest story of 2023, one topic shot straight to the top of the list. It can't have escaped anyone's notice that artificial intelligence rocketed to prominence this year, with OpenAI, the maker of ChatGPT, becoming a household name. Hundreds of millions of people are now using large language models on a regular basis, in a rapid roll-out of technology that is essentially unprecedented.

IT WAS hard to avoid artificial intelligence in 2023, with the vertiginous rise of chatbots powered by large language models (LLMs). By February, OpenAI's ChatGPT had become the fastest-growing app of all time. By the year's end, it had become an everything machine: browsing the internet, interpreting pictures, generating any requested image and inserting itself into many existing tools and services – and it wasn't the only AI to do so. "Where the technology is going next is clearly to be more multimodal,"…