to-control-ai-we-need-to-understand-more-about-humans

And among the things we urgently need to learn more about is not just how artificial intelligence works, but how humans work. Humans are also the only species to have developed "group normativity" – an elaborate system of rules and norms that designate what is collectively acceptable and not acceptable for other people to do, kept in check by group efforts to punish those who break the rules. But our complex normative social orders are less about ethical choices than they are about the coordination of billions of people making millions of choices on a daily basis about how to behave. Are we prepared for AIs that start building their own normative systems--their own rules about what is acceptable and unacceptable for a machine to do--in order to coordinate their own interactions?