Goto

Collaborating Authors

Hot papers on arXiv from the past month: August 2021

AIHub

Reproduced under a CC BY 4.0 license. Here are the most tweeted papers that were uploaded onto arXiv during August 2021. Results are powered by Arxiv Sanity Preserver. How to avoid machine learning pitfalls: a guide for academic researchers Michael A. Lones Submitted to arXiv on: 5 August 2021 Abstract: This document gives a concise outline of some of the common mistakes that occur when using machine learning techniques, and what can be done to avoid them. It is intended primarily as a guide for research students, and focuses on issues that are of particular concern within academic research, such as the need to do rigorous comparisons and reach valid conclusions.


ML and NLP Research Highlights of 2021

#artificialintelligence

In this post, I will cover the papers and research areas that I found most inspiring. I tried to cover the papers that I was aware of but likely missed many relevant ones. Feel free to highlight them as well as ones that you found inspiring in the comments. Pre-trained models were applied in many different domains and started to be considered critical for ML research [1]. In computer vision, supervised pre-trained models such as Vision Transformer [2] have been scaled up [3] and self-supervised pre-trained models have started to match their performance [4]. The latter have been scaled beyond the controlled environment of ImageNet to random collections of images [5]. In speech, new models have been built based on wav2vec 2.0 [6] such as W2v-BERT [7] as well as more powerful multilingual models such as XLS-R [8]. At the same time, we saw new unified pre-trained models for previously under-researched modality pairs such as for videos and language [9] as well as speech and language [10]. In vision and language, controlled studies shed new light on important components of such multi-modal models [11][12].


Risks of AI Foundation Models in Education

arXiv.org Artificial Intelligence

If the authors of a recent Stanford report (Bommasani et al., 2021) on the opportunities and risks of "foundation models" are to be believed, these models represent a paradigm shift for AI and for the domains in which they will supposedly be used, including education. Although the name is new (and contested (Field, 2021)), the term describes existing types of algorithmic models that are "trained on broad data at scale" and "fine-tuned" (i.e., adapted) for particular downstream tasks, and is intended to encompass large language models such as BERT or GPT-3 and computer vision models such as CLIP. Such technologies have the potential for harm broadly speaking (e.g., Bender et al., 2021), but their use in the educational domain is particularly fraught, despite the potential benefits for learners claimed by the authors. In section 3.3 of the Stanford report, Malik et al. argue that achieving the goal of providing education for all learners requires more efficient computational approaches that can rapidly scale across educational domains and across educational contexts, for which they argue foundation models are uniquely well-suited. However, evidence suggests that not only are foundation models not likely to achieve the stated benefits for learners, but their use may also introduce new risks for harm.


Reflections on Foundation Models

#artificialintelligence

Recently, we released our report on foundation models, launched the Stanford Center for Research on Foundation Models (CRFM) as part of the Stanford Institute for Human-Centered AI (HAI), and hosted a workshop to foster community-wide dialogue. Our work received an array of responses from a broad range of perspectives; some folks graciously shared their commentaries with us. We see open discourse as necessary for forging the right norms, best practices, and broader ecosystem around foundation models. In this blog post, we talk through why we believe these models are so important and clarify several points in relation to the community response. In addition, we support and encourage further community discussion of these complex issues; feel free to reach out at contact-crfm@stanford.edu.


Reflections on Foundation Models

#artificialintelligence

Our work received an array of responses from a broad range of perspectives; some folks graciously shared their commentaries with us. We see open discourse as necessary for forging the right norms, best practices, and broader ecosystem around foundation models. In this blog post, we talk through why we believe these models are so important and clarify several points in relation to the community response. In addition, we support and encourage further community discussion of these complex issues; feel free to reach out at [email protected]. We define foundation models as models trained on broad data (generally using self-supervision at scale) that can be adapted to a wide range of downstream tasks.