Trust large language models at your own peril

According to Meta, Galactica can "summarize academic papers, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more." But soon after its launch, it was pretty easy for outsiders to prompt the model to provide "scientific research" on the benefits of homophobia, anti-Semitism, suicide, eating glass, being white, or being a man. Meanwhile, papers on AIDS or racism were blocked. As my colleague Will Douglas Heaven writes in his story about the debacle: "Meta's misstep--and its hubris--show once again that Big Tech has a blind spot about the severe limitations of large language models." Not only was Galactica's launch premature, but it shows how insufficient AI researchers' efforts to make large language models safer have been. Meta might have been confident that Galactica outperformed competitors in generating scientific-sounding content.