What Are Foundation Models?

The mics were live and tape was rolling in the studio where the Miles Davis Quintet was recording dozens of tunes in 1956 for Prestige Records. When an engineer asked for the next song's title, Davis shot back, "I'll play it, and tell you what it is later." Like the prolific jazz trumpeter and composer, researchers have been generating AI models at a feverish pace, exploring new architectures and use cases. Focused on plowing new ground, they sometimes leave to others the job of categorizing their work. A team of more than a hundred Stanford researchers collaborated to do just that in a 214-page paper released in the summer of 2021.