Two days ago, an article quoting Hinton who was saying that backprop is not necessarily the way to go for AI, generated lots of very cool discussion on this sub-reddit (here). In this discussion I would like us to answer the question: what are the problems of backprop? Indeed, the article didn't really answer this question, only referencing shortly to the difficulties of unsupervised learning. Here's my initial input: Problems that seem to be intrinsic to backprop: Problems that currently pose lots of difficulties and we're not sure are possible with backprop: Do you agree with these?

The forecast is not all gloomy – artificial intelligence (AI), machine learning (ML) and automation are also expected to create jobs that will likely be much more interesting and creative than the repetitive tasks of the industrial age. According to Andrew McAfee, principal research scientist at MIT and co-director of the university's Initiative on the Digital Economy (IDE), AI amounts to, "the largest disruption in labor and the way we work," in generations. But as Joi Ito, director of the MIT Media Lab and moderator of a panel titled, "Putting AI to Work," put it, the fear that machines will become smarter than humans and take over the world is tempered by the reality that "they're stupid and they've already taken over the world." Seth Earley, CEO of Earley Information Science, while agreeing there will be, "an enormous amount of disruption," from AI, was more optimistic about retraining for the jobs of the future.

Exercising good judgement in difficult situations is a much tougher standard. Above all, ethics must be realistic, and in our real world, bad things happen. Abe is very specific: he means "biased" in a technical, statistical sense. Cathy O'Neil has frequently argued that secret algorithms and secret data models are the real danger.

While this will serve as an overview of Scott's essay, which you can read for further detail and mathematical insights, we will start by with Fortmann-Roe's verbatim definitions which are central to the piece: Error due to Bias: The error due to bias is taken as the difference between the expected (or average) prediction of our model and the correct value which we are trying to predict. Again, imagine you can repeat the entire model building process multiple times. Fortmann-Roe ends the section on over- and under-fitting by pointing to another of his great essays (Accurately Measuring Model Prediction Error), and then moving on to the highly-agreeable recommendation that "resampling based measures such as cross-validation should be preferred over theoretical measures such as Aikake's Information Criteria." I recommend reading Scott Fortmann-Roe's entire bias-variance tradeoff essay, as well as his piece on measuring model prediction error.

Recently, a conference on artificial intelligence, tantalizingly titled "Superintelligence: Science or Fiction?", was hosted by the Future of Life Institute, which works to promote "optimistic visions of the future". The conference offered a range of opinions on the subject from a variety of experts, including Elon Musk of Tesla Motors and SpaceX, futurist Ray Kurzweil, Demis Hassabis of Google's DeepMind, neuroscientist and author Sam Harris, philosopher Nick Bostrom, philosopher and cognitive scientist David Chalmers, Skype co-founder Jaan Tallinn, as well as computer scientists Stuart Russell and Bart Selman. The discussion was led by MIT cosmologist Max Tegmark. The conversation's topics centered on the future benefits and risks of artificial superintelligence, with everyone generally agreeing that it's only a matter of time before AI becomes paramount in our lives. Eventually, AI will surpass human intelligence, with the ensuing risks and transformations.

Third, the Recommendation engine of the Social Network, continued to suggest me thing from that domain, a domain in which I am not that interested, but I was skipping them, in the hope to get better suggestions on the domain I actually cared much more. That engagement, at the moment, means for AI to talk more about things that we talk most, to read more about things we read more, to watch more things that we used to watch in the past, and the AI thinks that it is doing a great job. Out world, physical or virtual (Internet) is dominated by Artificial Intelligence. It is about the time we start to look into if, to understand how our Algorithms and AI is Radicalizing humans on the altar on Engagement and Profits.

Twenty-six people participated, and 16 papers were presented. The first day was devoted to paper presentations and discussions. The second (half) day, a joint session was held with two other workshops: (1) Building, Maintaining, and Using Organizational Memories and (2) Intelligent Information Integration. The reason for the joint session was that in all three workshops, ontologies play a prominent role, and the goal was to bring together researchers working on related issues in different communities.

It is clear from these discussions that the notion of context is far from defined and is dependent in its interpretation on a cognitive science versus an engineering (or system building) point of view. In identifying the two points of view, this workshop permitted us to go one step further than previous workshops (notably Maskery and Meads [1992] and Maskery, Hopkins, and Dudley [1992]). Once a distinction is made on the viewpoint, one can achieve a surprising consensus on the aspects of context that the workshop addressed -- mainly, the position, the elements, the representation, and the use of context. Despite this consensus on the aspects of context, agreement on the definition of context was not yet achieved.

A panel session at the 1989 International Joint Conference on Artificial Intelligence in Los Angeles dealt with the subject of knowledge-based systems; the session was entitled "Expert Systems: How Far Can They Go?" The panelists included Randall Davis (Massachusetts Institute of Technology); Stuart Dreyfus (University of California at Berkeley); Brian Smith (Xerox Palo Alto Research Center); and Terry Winograd (Stanford University), chairman. Part 1 of this article, which appeared in the Spring 1989 issue, began with Winograd's original charge to the panel, followed by lightly edited transcripts of presentations from Winograd and Dreyfus. Part 2 begins with the presentations from Smith and Davis and concludes with the panel discussion.