The Danger Of Advanced Artificial Intelligence Controlling Its Own Feedback - Liwaiwai

How would an artificial intelligence (AI) decide what to do? One common approach in AI research is called "reinforcement learning". Reinforcement learning gives the software a "reward" defined in some way, and lets the software figure out how to maximise the reward. This approach has produced some excellent results, such as building software agents that defeat humans at games like chess and Go, or creating new designs for nuclear fusion reactors. However, we might want to hold off on making reinforcement learning agents too flexible and effective.