Future of AI VI. Discussion of 'Superintelligence: Paths, Dangers, Strategies'

This post is a discussion of Nick Bostrom's book "Superintelligence". The book has had an effect on the thinking of many of the world's thought leaders. In that light, and given this series of blog posts is about the "Future of AI", it seemed important to read the book and discuss his ideas. In an ideal world, this post would certainly have contained more summaries of the books arguments and perhaps a later update will improve on that aspect. For the moment the review focuses on counter-arguments and perceived omissions (the post already got too long with just covering those). Bostrom considers various routes we have to forming intelligent machines and what the possible outcomes might be from developing such technologies. He is a professor of philosophy but has an impressive array of background degrees in areas such as mathematics, logic, philosophy and computational neuroscience. So let's start at the beginning and put the book in context by trying to understand what is meant by the term "superintelligence" In common with many contributions to the debate on artificial intelligence, Bostrom never defines what he means by intelligence. Obviously, this can be problematic. On the other hand, superintelligence is defined as outperforming humans in every intelligent capability that they express. Personally, I've developed the following definition of intelligence: "Use of information to take decisions which save energy". Here by information I might mean data or facts or rules, and by saving energy I mean saving'free' energy.1 However, accepting Bostrom's lack of definition of intelligence (and perhaps taking note of my own), we can still consider the routes to superintelligence Bostrom proposes.