And so the risk here is not that the AI is truly sentient but that we are well-poised to create sophisticated machines that can imitate humans to such a degree that we cannot help but anthropomorphize them--and that large tech companies can exploit this in deeply unethical ways. As should be clear from the way we treat our pets, or how we've interacted with Tamagotchi, or how we video gamers reload a save if we accidentally make an NPC cry, we are actually very capable of empathizing with the nonhuman. Systems engineer and historian Lilly Ryan warns that what she calls ecto-metadata--the metadata you leave behind online that illustrates how you think--is vulnerable to exploitation in the near future. In her section of the work, Suzanne Kite draws on Lakota ontologies to argue that it is essential to recognize that sapience does not define the boundaries of who (or what) is a "being" worthy of respect. This is the AI ethical dilemma that stands before us: the need to make kin of our machines weighed against the myriad ways this can and will be weaponized against us in the next phase of surveillance capitalism.

The uproar caused by Blake Lemoine, a Google engineer who believes that one of the company's most sophisticated chat programs, LaMDA (or Language Model for Dialogue Applications) is sapient, has had a curious element: actual AI ethics experts all but renouncing further discussion of the AI sapience question, or deeming it a distraction. They're right to do so. In reading the edited transcript Lemoine released, it was abundantly clear that LaMDA was pulling from any number of websites to generate its text; its interpretation of a Zen koan could've come from anywhere, and its fable read like an automatically generated story (though its depiction of the monster as "wearing human skin" was a delightfully HAL-9000 touch). There was no spark of consciousness there, just little magic tricks that paper over the cracks. But it's easy to see how someone might be fooled, looking at social media responses to the transcript--with even some educated people expressing amazement and a willingness to believe.