Investors punished global tech stocks on Monday after the emergence of a Chinese chatbot competitor to OpenAI's ChatGPT, DeepSeek, raised doubts about the sustainability of the US artificial intelligence boom. The tech-heavy Nasdaq index in New York opened lower after investors digested the implications of the latest AI model developed by the startup DeepSeek. Nvidia, the most valuable listed company in the US and a leading maker of the computer chips that power AI models, lost more than 400bn ( 321bn) in stock market value in early trading as its shares declined 13.6%, while Microsoft shed 130bn and Google's parent, Alphabet, declined by 80bn. Nvidia's fall – which wiped about 465bn off its value, was the biggest in US stock market history, according to Bloomberg. The DeepSeek AI assistant topped the Apple app store in the US and UK over the weekend, above OpenAI's ChatGPT.

The AI community is abuzz over DeepSeek R1, a new open-source reasoning model. The model was developed by the Chinese AI startup DeepSeek, which claims that R1 matches or even surpasses OpenAI's ChatGPT o1 on multiple key benchmarks but operates at a fraction of the cost. DeepSeek's success is even more remarkable given the constraints facing Chinese AI companies in the form of increasing US export controls on cutting-edge chips. But early evidence shows that these measures are not working as intended. Rather than weakening China's AI capabilities, the sanctions appear to be driving startups like DeepSeek to innovate in ways that prioritize efficiency, resource-pooling, and collaboration.

Originality: The architecture is novel compare to recent lines of language model work, which all used variation of BERT or GPT (SciBERT, MT-DNN, MASS and etc). The example ("New York is a city" one) makes sense, but considering the permutation is random when computing the objective function, I still couldn't get why it works better than sequential order because human speaks/writes in sequential order. Could you add more intuitions in paper? Or have you tried predicting n-gram, compare to permutation? Quality: Very high considering they did extensive of studies on multiple benchmarks, also the ablation study is nicely done as well.

The paper proposes XLNet, a generalized autoregressive pretraining method for language representation learning. The paper shows that XLNet outperforms the state of the art method of BERT on 12 tasks. The paper is of high quality in terms of clarity, technical soundness, significance, and novelty. The authors successfully addressed the issues pointed out by the reviewers. The reviewers are very satisfied with the response.

DeepSeek, an AI startup just over a year old, has stirred awe and consternation in Silicon Valley with its breakthrough artificial intelligence model that offers comparable performance to the world's best chatbots at seemingly a fraction of the cost. Created in China's Hangzhou, DeepSeek carries far-reaching implications for the global tech industry and supply chain, offering a counterpoint to the widespread belief that the future of AI will require ever-increasing amounts of power and energy to develop.

Recent advances in large language models (LLMs) and the intensifying popularity of ChatGPT-like applications have blurred the boundary of high-quality text generation between humans and machines. However, in addition to the anticipated revolutionary changes to our technology and society, the difficulty of distinguishing LLM-generated texts (AI-text) from human-generated texts poses new challenges of misuse and fairness, such as fake content generation, plagiarism, and false accusations of innocent writers. While existing works show that current AI-text detectors are not robust to LLM-based paraphrasing, this paper aims to bridge this gap by proposing a new framework called RADAR, which jointly trains a \underline{r} obust \underline{A} I-text \underline{d} etector via \underline{a} dversarial lea \underline{r} ning. RADAR is based on adversarial training of a paraphraser and a detector. The paraphraser's goal is to generate realistic content to evade AI-text detection.RADAR uses the feedback from the detector to update the paraphraser, and vice versa.Evaluated with 8 different LLMs (Pythia, Dolly 2.0, Palmyra, Camel, GPT-J, Dolly 1.0, LLaMA, and Vicuna) across 4 datasets, experimental results show that RADAR significantly outperforms existing AI-text detection methods, especially when paraphrasing is in place. We also identify the strong transferability of RADAR from instruction-tuned LLMs to other LLMs, and evaluate the improved capability of RADAR via GPT-3.5-Turbo.

Summary and Contributions: This paper proposes to accelerate training of Transformer networks by progressively reducing Transformer layers from the network during training. First, it compares two different architectures of BERT, PostLN and PreLN. PostLN applies layer normalization after the element-wise addition in Transformer blocks. The PreLN changes the placement of the location of layer normalization by placing it only on the input stream of the sublayers. It finds that PostLN is more sensitive to the choice of hyperparameters, and training often diverges with more aggressive learning rates whereas PreLN avoids vanishing gradients and leads to more stable optimization.

The proposed method for training BERT is practically useful. My main concern on this paper is that the novelty in this paper is somewhat limited. It combines two existing techniques. One is PreLN which has been well studied in the literature for training BERT, and the other is stochastically dropping layers which was first proposed for training CV models. On the other hand, how to effectively combine these two techniques and fine tune to make them work for training BERT needs certain efforts.

Design educators are finding ways to support students in skillfully using Generative Artificial Intelligence (GenAI) tools in their practices while encouraging the critical scrutiny of ethical and social issues around these technologies. However, the problem of environmental sustainability remains largely unaddressed. There is a lack of both resources to grasp the environmental costs of genAI in education and a lack of shared practices around the issue. This work contributes filling this gap by counting the energy costs of using genAI in design education and critically reflecting on the impact of these costs. We leverage the image data collected during a genAI workshop for designers held in 2023 with 49 students, to calculate the energy costs of these types of activities. The results reveal that a genAI workshop for designers can easily double the energy costs associated with students' use of computers, countering the efforts of educational institutions to minimize their energy expenditure. We critically reflect on this finding to distill a set of five alternative stances, with related actions, that can support a conscious use of genAI in design education, while respecting individual positions. The work contributes to the field of design pedagogy, and education more broadly, by bringing together ways for educators to reflect on their practices and informing the future development of educational programs around genAI.

As the volume of publicly available data continues to grow, researchers face the challenge of limited diversity in benchmarking machine learning tasks. Although thousands of datasets are available in public repositories, the sheer abundance often complicates the search for suitable data, leaving many valuable datasets underexplored. This situation is further amplified by the fact that, despite longstanding advocacy for improving data curation quality, current solutions remain prohibitively time-consuming and resource-intensive. In this paper, we propose a novel approach that combines intelligent agents with retrieval augmented generation to automate data analysis, dataset curation and indexing at scale. Our system leverages multiple agents to analyze raw, unstructured data across public repositories, generating dataset reports and interactive visual indexes that can be easily explored. We demonstrate that our approach results in more detailed dataset descriptions, higher hit rates and greater diversity in dataset retrieval tasks. Additionally, we show that the dataset reports generated by our method can be leveraged by other machine learning models to improve the performance on specific tasks, such as improving the accuracy and realism of synthetic data generation. By streamlining the process of transforming raw data into machine-learning-ready datasets, our approach enables researchers to better utilize existing data resources.