Chat models, such as ChatGPT, have shown impressive capabilities and have been rapidly adopted across numerous domains. However, these models are only accessible through a restricted API, creating barriers for new research and progress in the field. We propose a pipeline that can automatically generate a high-quality multi-turn chat corpus by leveraging ChatGPT to engage in a conversation with itself. Subsequently, we employ parameter-efficient tuning to enhance LLaMA, an open-source large language model. The resulting model, named Baize, demonstrates good performance in multi-turn dialogues with guardrails that minimize potential risks. Furthermore, we propose a new technique called Self-Distill with Feedback, to further improve the performance of the Baize models with feedback from ChatGPT. The Baize models and data are released for research purposes only at https://github.com/project-baize/baize-chatbot. An online demo is also available at https://huggingface.co/spaces/project-baize/chat-with-baize.

Zero-shot learning (ZSL) is a promising approach to generalizing a model to categories unseen during training by leveraging class attributes, but challenges remain. Recently, methods using generative models to combat bias towards classes seen during training have pushed state of the art, but these generative models can be slow or computationally expensive to train. Also, these generative models assume that the attribute vector of each unseen class is available a priori at training, which is not always practical. Additionally, while many previous ZSL methods assume a one-time adaptation to unseen classes, in reality, the world is always changing, necessitating a constant adjustment of deployed models. Models unprepared to handle a sequential stream of data are likely to experience catastrophic forgetting. We propose a Meta-learned Attribute self-Interaction Network (MAIN) for continual ZSL. By pairing attribute self-interaction trained using meta-learning with inverse regularization of the attribute encoder, we are able to outperform state-of-the-art results without leveraging the unseen class attributes while also being able to train our models substantially faster (>100x) than expensive generative-based approaches. We demonstrate this with experiments on five standard ZSL datasets (CUB, aPY, AWA1, AWA2, and SUN) in the generalized zero-shot learning and continual (fixed/dynamic) zero-shot learning settings. Extensive ablations and analyses demonstrate the efficacy of various components proposed.

Sam Altman has appeared to lead credence to the theory he was fired from OpenAI over his company's super powerful, secret new AI system he helped build. Multiple employees reportedly warned the company's board of directors that this project, named Q* (pronounced'Q star'), was becoming so advanced it could already pass math exams and perform critical thinking tasks. And they felt Altman was not taking their warnings seriously. In an interview this week, Altman did not deny the existence of the secret program that some employees said was responsible for his firing. Instead, he called the revelation of Q* an'unfortunate leak.' Altman was fired, then hired by OpenAI investor Microsoft, and then re-hired by OpenAI - which also gave the boot to most of the board that cut Altman loose - all over the course of just five days in November.

No human celebrating a first birthday is as verbose, knowledgeable, or prone to fabrication as ChatGPT, which is blowing out its first candle as I type these words. Of course, OpenAI's game-changing large language model was precocious at birth, tumbling into civilization's ongoing conversation like an uninvited guest busting into a dinner party and instantly commanding the room. The chatbot astonished everyone who prompted it with fully realized, if not always completely factual, responses to almost any possible query. Suddenly, the world had access to a Magic 8 Ball with a PhD in every discipline. In almost no time, 100 million people became regular users, delighted and terrified to realize that humans had suddenly lost their monopoly on discourse.

In the surprise ouster and restoration of Sam Altman as chief executive officer at OpenAI, only one person, Adam D'Angelo, managed to play a role on each side of the drama. D'Angelo, a former Facebook executive and founder of the question-and-answer platform Quora, was one of four members of the board who fired Altman, and the sole surviving director named to a new board of the artificial-intelligence company that took over on Wednesday.

Microsoft is significantly expanding the functions of Paint in Windows 11. The app is also getting a new version. The outdated program is to become a modern image editor that also contains AI functions. In the future, you will be able to use the OpenAI-LLM Dall-E directly in Windows 11 and in Paint. The new functions are also available after installing the Microsoft Paint app from the App Store.

At around 11:30 a.m. on the Friday before Thanksgiving, Microsoft's chief executive, Satya Nadella, was having his weekly meeting with senior leaders when a panicked colleague told him to pick up the phone. An executive from OpenAI, an artificial-intelligence startup into which Microsoft had invested a reported thirteen billion dollars, was calling to explain that within the next twenty minutes the company's board would announce that it had fired Sam Altman, OpenAI's C.E.O. and co-founder. It was the start of a five-day crisis that some people at Microsoft began calling the Turkey-Shoot Clusterfuck. Nadella has an easygoing demeanor, but he was so flabbergasted that for a moment he didn't know what to say. He'd worked closely with Altman for more than four years and had grown to admire and trust him.

Artificial intelligence seems predestined to become a bigger part of our lives. To what extent is the A.I. push being led by Sam Altman and the OpenAI team a cause for concern? If you enjoy this show, please consider signing up for Slate Plus. Slate Plus members get benefits like zero ads on any Slate podcast, bonus episodes of shows like Slow Burn and Dear Prudence--and you'll be supporting the work we do here on What Next TBD. Sign up now at slate.com/whatnextplus to help support our work.

The introduction of large public legal datasets has brought about a renaissance in legal NLP. Many of these datasets are comprised of legal judgements - the product of judges deciding cases. This fact, together with the way machine learning works, means that several legal NLP models are models of judges. While some have argued for the automation of judges, in this position piece, we argue that automating the role of the judge raises difficult ethical challenges, in particular for common law legal systems. Our argument follows from the social role of the judge in actively shaping the law, rather than merely applying it. Since current NLP models come nowhere close to having the facilities necessary for this task, they should not be used to automate judges. Furthermore, even in the case the models could achieve human-level capabilities, there would still be remaining ethical concerns inherent in the automation of the legal process.

We present an approximate attention mechanism named HyperAttention to address the computational challenges posed by the growing complexity of long contexts used in Large Language Models (LLMs). Recent work suggests that in the worst-case scenario, quadratic time is necessary unless the entries of the attention matrix are bounded or the matrix has low stable rank. We introduce two parameters which measure: (1) the max column norm in the normalized attention matrix, and (2) the ratio of row norms in the unnormalized attention matrix after detecting and removing large entries. We use these fine-grained parameters to capture the hardness of the problem. Despite previous lower bounds, we are able to achieve a linear time sampling algorithm even when the matrix has unbounded entries or a large stable rank, provided the above parameters are small. HyperAttention features a modular design that easily accommodates integration of other fast low-level implementations, particularly FlashAttention. Empirically, employing Locality Sensitive Hashing (LSH) to identify large entries, HyperAttention outperforms existing methods, giving significant speed improvements compared to state-of-the-art solutions like FlashAttention. We validate the empirical performance of HyperAttention on a variety of different long-context length datasets. For example, HyperAttention makes the inference time of ChatGLM2 50\% faster on 32k context length while perplexity increases from 5.6 to 6.3. On larger context length, e.g., 131k, with causal masking, HyperAttention offers 5-fold speedup on a single attention layer.