Media
This Is How I Used Artificial Intelligence in My Life During the Last 24 Hours
What can we do in 24 hours? What happens in our lives between sunrise and sunset? What happens in 24 hours around the world? On average, in 24 hours, I will experience 104,000 heartbeats, I'll take a breath about 23,000 times, I'll walk about 8,000 steps on average, and in the shower, I'll spend about 12 minutes. My body will shed and create up to 50 trillion new cells, and I usually spend 20 minutes in the bathroom. There will be a 0.35 mm growth in my hair, and I will also lose somewhere between 40 and 100 hairs at the same time, and on average, I'll speak for roughly 48,000 words.
Few-shot Adaptation Works with UnpredicTable Data
Chan, Jun Shern, Pieler, Michael, Jao, Jonathan, Scheurer, Jérémy, Perez, Ethan
Prior work on language models (LMs) shows that training on a large number of diverse tasks improves few-shot learning (FSL) performance on new tasks. We take this to the extreme, automatically extracting 413,299 tasks from internet tables - orders of magnitude more than the next-largest public datasets. Finetuning on the resulting dataset leads to improved FSL performance on Natural Language Processing (NLP) tasks, but not proportionally to dataset scale. In fact, we find that narrow subsets of our dataset sometimes outperform more diverse datasets. For example, finetuning on software documentation from support.google.com raises FSL performance by a mean of +7.5% on 52 downstream tasks, which beats training on 40 human-curated NLP datasets (+6.7%). Finetuning on various narrow datasets leads to similar broad improvements across test tasks, suggesting that the gains are not from domain adaptation but adapting to FSL in general. We do not observe clear patterns between the datasets that lead to FSL gains, leaving open questions about why certain data helps with FSL.