In this work, we revisit linguistic acceptability in the context of large language models. We introduce CoLAC - Corpus of Linguistic Acceptability in Chinese, the first large-scale acceptability dataset for a non-Indo-European language. It is verified by native speakers and is the first acceptability dataset that comes with two sets of labels: a linguist label and a crowd label. Our experiments show that even the largest InstructGPT model performs only at chance level on CoLAC, while ChatGPT's performance (48.30 MCC) is also much below supervised models (59.03 MCC) and human (65.11 MCC). Through cross-lingual transfer experiments and fine-grained linguistic analysis, we provide detailed analysis of the model predictions and demonstrate for the first time that knowledge of linguistic acceptability can be transferred across typologically distinct languages, as well as be traced back to pre-training. Our dataset is publicly available at \url{https://github.com/huhailinguist/CoLAC}.

Large language models (LLMs) have shown remarkable capabilities in language understanding and generation. However, such impressive capability typically comes with a substantial model size, which presents significant challenges in both the deployment, inference, and training stages. With LLM being a general-purpose task solver, we explore its compression in a task-agnostic manner, which aims to preserve the multi-task solving and language generation ability of the original LLM. One challenge to achieving this is the enormous size of the training corpus of LLM, which makes both data transfer and model post-training over-burdensome. Thus, we tackle the compression of LLMs within the bound of two constraints: being task-agnostic and minimizing the reliance on the original training dataset. Our method, named LLM-Pruner, adopts structural pruning that selectively removes non-critical coupled structures based on gradient information, maximally preserving the majority of the LLM's functionality. To this end, the performance of pruned models can be efficiently recovered through tuning techniques, LoRA, in merely 3 hours, requiring only 50K data. We validate the LLM-Pruner on three LLMs, including LLaMA, Vicuna, and ChatGLM, and demonstrate that the compressed models still exhibit satisfactory capabilities in zero-shot classification and generation. The code is available at: https://github.com/horseee/LLM-Pruner

Large language models (LLMs) can be used to generate smaller, more refined datasets via few-shot prompting for benchmarking, fine-tuning or other use cases. However, understanding and evaluating these datasets is difficult, and the failure modes of LLM-generated data are still not well understood. Specifically, the data can be repetitive in surprising ways, not only semantically but also syntactically and lexically. We present LinguisticLens, a novel inter-active visualization tool for making sense of and analyzing syntactic diversity of LLM-generated datasets. LinguisticLens clusters text along syntactic, lexical, and semantic axes. It supports hierarchical visualization of a text dataset, allowing users to quickly scan for an overview and inspect individual examples. The live demo is available at shorturl.at/zHOUV.

Recent work has demonstrated substantial gains in pre-training large-language models (LLMs) followed by supervised fine-tuning on the downstream task. In this paper, we evaluate the performance of the GPT-neo model using $6$ commonsense reasoning benchmark tasks. We aim to examine the performance of smaller models using the GPT-neo models against several larger model baselines such as GPT-$3$, Llama-$2$, MPT and Falcon. Upon fine-tuning with the appropriate set of hyperparameters, our model achieves competitive accuracy on several tasks. We also investigate and substantiate our results using attention-head visualization to better understand the model performance. Finally, we conduct various robustness tests using various methods to gauge the model performance under numerous settings.

Joe Biden showed up on the United Auto Workers' picket line today--but even with the president's historic gesture of union support, a specter is looming. The shift to electric vehicles is coming, and "this future is not guaranteed to offer the same kinds of middle-class jobs and robust benefits that unionized autoworkers enjoy in many states," Nitish Pahwa writes. He takes a close look at what the EV transition is going to mean for organized labor. Fred Kaplan noticed three of his own books among the list of titles that Meta used to train its new large language model, LLaMA (basically its answer to ChatGPT). So he decided to ask it some questions--what did it think of his books?

The CIA is set to launch its own ChatGPT-style AI tool to help sift through mountains of data for clues in ongoing investigations. Intended to mirror the famed OpenAI tech, the Central Intelligence Agency's latest initiative will use artificial intelligence to help analysts better access open-source intelligence, agency officials said. The CIA's Open Source Enterprise division developed the tech, which is also intended to be rolled out across the US government's 18 intelligence agencies in an effort to rival China's growing intelligence capabilities. 'We've gone from newspapers and radio, to newspapers and television, to newspapers and cable television, to basic internet, to big data, and it just keeps going,' said Randy Nixon, director of the CIA's AI division. Nixon noted that analyzing the level of data across the web is a significant challenge that the AI program would help handle, adding: 'We have to find the needles in the needle field.'

OpenAI is talking to investors about a possible share sale that would value the artificial-intelligence startup behind ChatGPT at between $80 billion to $90 billion, almost triple its level earlier this year, people familiar with the discussions said.

OpenAI is rolling out the ability to carry on conversations with a human-sounding robot on the ChatGPT app. Alexa and Siri are about to get really jealous. The voice technology smart speakers are being taken on by a full-fledged humanoid AI robot being rolled out on the ChatGPT app for Plus paying customers. Starting this week, a new feature will be available on the iOS and Google Play ChatGPT apps that could potentially eliminate the need for keyboards. Let's dive in and see exactly what is going to be at our fingertips.

The quickest way to second-guess a decision to major in English is this: have an extended family full of Salvadoran immigrants and pragmatic midwesterners. The ability to recite Chaucer in the original Middle English was unlikely to land me a job that would pay off my student loans and help me save for retirement, they suggested when I was a college freshman still figuring out my future. I stuck with English, but when my B.A. eventually spat me out into the thick of the Great Recession, I worried that they'd been right. After all, computer-science degrees, and certainly not English, have long been sold to college students as among the safest paths toward 21st-century job security. Coding jobs are plentiful across industries, and the pay is good--even after the tech layoffs of the past year.

Amazon's bid for AI glory is in the billions. It's investing up to $4 billion in OpenAI rival Anthropic to provide advanced deep learning and other services for its Amazon Web Services (AWS) customers. Google has already invested $400 million in the company, which was founded by former OpenAI executives. Anthropic recently unveiled its first consumer-facing chatbot Claude 2, accessible by subscription much like OpenAI's ChatGPT. The Claude Constitutional AI system is guided by 10 "foundational" principles of fairness and autonomy and is supposed to be harder to trick than other AI. Anthropic is already working on a chatbot it calls Claude-Next, which is supposed to be 10 times more powerful than any current AI.