We need to talk about how artificial intelligence can manipulate humans

We have all read about artificial intelligence becoming smarter than us, a future in which we become like pets and can only hope AI will be benevolent. My experience watching tens of millions of interactions between humans and artificial conversational agents, or bots, has convinced me there are far more immediate risks--as well as tremendous opportunities. From 2007 to 2014 I was CEO of Cognea, which offered a platform to rapidly build complex virtual agents, using a combination of structured and deep learning. It was used by tens of thousands of developers, including half a dozen Fortune 100 companies, and acquired by IBM Watson in 2014. As I studied how people interacted with the tens of thousands of agents built on our platform, it became clear that humans are far more willing than most people realize to form a relationship with AI software.