In 2024, increased adoption of biometric surveillance systems, such as the use of AI-powered facial recognition in public places and access to government services, will spur biometric identity theft and anti-surveillance innovations. Individuals aiming to steal biometric identities to commit fraud or gain access to unauthorized data will be bolstered by generative AI tools and the abundance of face and voice data posted online. Already, voice clones are being used for scams. Take for example, Jennifer DeStefano, a mom in Arizona who heard the panicked voice of her daughter crying "Mom, these bad men have me!" after receiving a call from an unknown number. DeStefano was eventually able to confirm that her daughter was safe.

An Arizona family was terrified a few months ago when what they thought was a kidnapping and ransom call turned out to be a total scam created by artificial intelligence. As reports grow of scam calls that sound identical to loved ones, many fear that AI could be weaponized to threaten people with technology that's easy to access and only requires a small fee, several minutes and a stable internet connection. Jennifer DeStefano received an anonymous call one January afternoon while her 15-year-old daughter was out of town for a ski race. DeStefano heard her daughter answer the phone, panicking and screaming, quickly followed by a man's voice threatening to drug and kidnap DeStefano's daughter unless he was sent $1 million, CNN reported. DeStefano was able to reach her daughter a few minutes later, who was fine and puzzled about what had happened, because she hadn't been abducted and wasn't involved in the ransom call.

This month, a local TV-news station in Arizona ran an unsettling report: A mother named Jennifer DeStefano says that she picked up the phone to the sound of her 15-year-old crying out for her, and was asked to pay a $1 million ransom for her daughter's return. In reality, the teen had not been kidnapped, and was safe; DeStefano believes someone used AI to create a replica of her daughter's voice to deploy against her family. "It was completely her voice," she said in one interview. It was the way she would have cried." DeStefano's story has since been picked up by other outlets, while similar stories of AI voice scams have surfaced on TikTok and been reported by The Washington Post.

An Arizona mother detailed a terrifying phone scam that faked her 15-year-old daughter’s voice using artificial intelligence to claim she was kidnapped.