Future of AI 6. Discussion of 'Superintelligence: Paths, Dangers, Strategies'

Update: readers of the post have also pointed out this critique by Ernest Davis and this response to Davis by Rob Bensinger. Update 2: Both Rob Bensinger and Michael Tetelman rightly pointed out that my intelligence definition was sloppily defined. I've added a clarification that the defintion is'for a given task'. This post is a discussion of Nick Bostrom's book "Superintelligence". The book has had an effect on the thinking of many of the world's thought leaders. In that light, and given this series of blog posts is about the "Future of AI", it seemed important to read the book and discuss his ideas. In an ideal world, this post would certainly have contained more summaries of the books arguments and perhaps a later update will improve on that aspect. For the moment the review focuses on counter-arguments and perceived omissions (the post already got too long with just covering those). Bostrom considers various routes we have to forming intelligent machines and what the possible outcomes might be from developing such technologies. He is a professor of philosophy but has an impressive array of background degrees in areas such as mathematics, logic, philosophy and computational neuroscience. So let's start at the beginning and put the book in context by trying to understand what is meant by the term "superintelligence" In common with many contributions to the debate on artificial intelligence, Bostrom never defines what he means by intelligence. Obviously, this can be problematic. On the other hand, superintelligence is defined as outperforming humans in every intelligent capability that they express.