Artificial Intelligence and the King Midas Problem - Future of Life Institute

It's a phrase that often pops up in discussions about the safety and ethics of artificial intelligence. How can scientists create AI with goals and values that align with those of the people it interacts with? Very simple robots with very constrained tasks do not need goals or values at all. Although the Roomba's designers know you want a clean floor, Roomba doesn't: it simply executes a procedure that the Roomba's designers predict will work--most of the time. If your kitten leaves a messy pile on the carpet, Roomba will dutifully smear it all over the living room.