Goto

Collaborating Authors

The Paradigm Shift of Self-Supervised Learning

#artificialintelligence

"If intelligence was a cake, unsupervised learning would be the cake, supervised learning would be the icing on the cake, and reinforcement learning would be the cherry on the cake. We know how to make the icing and the cherry, but we don't know how to make the cake." By 2016, Yann LeCun began to hedge with his use of the term "unsupervised learning". In NIPS 2016, he started to call it in even more nebulous terms "predictive learning": I have always had trouble with the use of the term "Unsupervised Learning". In 2017, I had predicted that Unsupervised Learning will not progress much and said "there seems to be a massive conceptual disconnect as to how exactly it should work" and that it was the "dark matter" of machine learning.


Self-supervised learning is the future of AI

#artificialintelligence

Despite the huge contributions of deep learning to the field of artificial intelligence, there's something very wrong with it: It requires huge amounts of data. This is one thing that both the pioneers and critics of deep learning agree on. In fact, deep learning didn't emerge as the leading AI technique until a few years ago because of the limited availability of useful data and the shortage of computing power to process that data. Reducing the data-dependency of deep learning is currently among the top priorities of AI researchers. In his keynote speech at the AAAI conference, computer scientist Yann LeCun discussed the limits of current deep learning techniques and presented the blueprint for "self-supervised learning," his roadmap to solve deep learning's data problem.


Self-supervised learning: The plan to make deep learning data-efficient

#artificialintelligence

This article is part of Demystifying AI, a series of posts that (try to) disambiguate the jargon and myths surrounding AI. Despite the huge contributions of deep learning to the field of artificial intelligence, there's something very wrong with it: It requires huge amounts of data. This is one thing that both the pioneers and critics of deep learning agree on. In fact, deep learning didn't emerge as the leading AI technique until a few years ago because of the limited availability of useful data and the shortage of computing power to process that data. Reducing the data-dependency of deep learning is currently among the top priorities of AI researchers.


Self-supervised learning is the future of AI

#artificialintelligence

Despite the huge contributions of deep learning to the field of artificial intelligence, there's something very wrong with it: It requires huge amounts of data. This is one thing that both the pioneers and critics of deep learning agree on. In fact, deep learning didn't emerge as the leading AI technique until a few years ago because of the limited availability of useful data and the shortage of computing power to process that data. Reducing the data-dependency of deep learning is currently among the top priorities of AI researchers. In his keynote speech at the AAAI conference, computer scientist Yann LeCun discussed the limits of current deep learning techniques and presented the blueprint for "self-supervised learning," his roadmap to solve deep learning's data problem.


The Illustrated Self-Supervised Learning

#artificialintelligence

Yann Lecun, in his talk, introduced the "cake analogy" to illustrate the importance of self-supervised learning. Though the analogy is debated(ref: Deep Learning for Robotics(Slide 96), Pieter Abbeel), we have seen the impact of self-supervised learning in the Natural Language Processing field where recent developments (Word2Vec, Glove, ELMO, BERT) have embraced self-supervision and achieved state of the art results. "If intelligence is a cake, the bulk of the cake is self-supervised learning, the icing on the cake is supervised learning, and the cherry on the cake is reinforcement learning (RL)." Curious to know how self-supervised learning has been applied in the computer vision field, I read up on existing literature on self-supervised learning applied to computer vision through a recent survey paper by Jing et. This post is my attempt to provide an intuitive visual summary of the patterns of problem formulation in self-supervised learning.