What's the Deal With Artificial Intelligence Killing Humans?

Some, such as the Machine Intelligence Research Institute's Eliezer Yudkowsky, have proposed that we need to build A.I. that is friendly to humans. Thanks to the complexities of machine learning, computers can come at their tasks in unpredictable ways, potentially creating new problems thanks to their single-minded focus on finding novel solutions to existing ones. At MIRI, this is known as the "alignment problem"--the discrepancy between the ways machines complete tasks and what's actually good for the humans who assigned those tasks in the first place. Much as you'd want to understand some basic structural engineering principles before trying to build a bridge, MIRI holds that we need to pursue research into the mathematical underpinnings of alignment itself before we go about actually creating superintelligent systems.