With the release of large language models (LLMs) like ChatGPT - a question-answering chatbot - and Galactica - a tool for scientific writing - comes a new wave of an old conversation about what these models can do. Their capabilities have been presented as extraordinary, mind-blowing, autonomous; at its peak, fascinated evangelists claimed that these models contain "humanity's scientific knowledge", are approaching Artificial General Intelligence (AGI), and even resemble consciousness. However, such hype is not much more than a distraction from the actual harm perpetuated by these systems. People get hurt from the very practical ways such models fall short in deployment, and these failures are the result of choices made by the builders of these systems - choices we are obliged to critique and hold model builders accountable for. Among the most celebrated AI deployments is the use of BERT - one of the first large language models developed by Google - to improve Google search engine results.