Goto

Collaborating Authors

constraint


Simultaneous clustering and representation learning

AIHub

The success of deep learning over the last decade, particularly in computer vision, has depended greatly on large training data sets. Even though progress in this area boosted the performance of many tasks such as object detection, recognition, and segmentation, the main bottleneck for future improvement is more labeled data. Self-supervised learning is among the best alternatives for learning useful representations from the data. In this article, we will briefly review the self-supervised learning methods in the literature and discuss the findings of a recent self-supervised learning paper from ICLR 2020 [14]. We may assume that most learning problems can be tackled by having clean labeling and more data obtained in an unsupervised way.


Permanent beta: Why frugal innovation is such a hot concept for entrepreneurs

ZDNet

The concept of frugal innovation originated in emerging markets, where social entrepreneurs and enthusiastic designers perfected the idea of creating low-cost, highly user-friendly devices that also fulfil a social need. A clay fridge that uses no electricity but keeps food cool, mobile money services for people without bank accounts like M-PESA, and a billboard that collects water from humid air in a rain-scarce area of Peru are all cited as examples of frugal innovation in developing markets. But increasingly the idea is being used more broadly. The recent COVID-19 outbreak has shown just how far frugal innovation can take off in developed markets: companies, healthcare organisations and entrepreneurs were faced with a real problem to tackle in a short amount of time with unexpectedly limited resources. That resulted in innovations like PPE that could be 3D printed at home or made from scuba masks, ventilators hacked with readily available equipment to double their capacity, and companies sharing the designs for their kit to allow other organisations to manufacture it themselves.


"What's that? Reinforcement Learning in the Real-world?"

#artificialintelligence

Reinforcement Learning offers a distinctive way of solving the Machine Learning puzzle. It's sequential decision-making ability, and suitability to tasks requiring a trade-off between immediate and long-term returns are some components that make it desirable in settings where supervised-learning or unsupervised learning approaches would, in comparison, not fit as well. By having agents start with zero knowledge then learn qualitatively good behaviour through interaction with the environment, it's almost fair to say Reinforcement Learning (RL) is the closest thing we have to Artificial General Intelligence yet. We can see RL being used in robotics control, treatment design in healthcare, among others; but why aren't we boasting of many RL agents being scaled up to real-world production systems? There's a reason why games, like Atari, are such nice RL benchmarks -- they let us care only about maximizing the score and not worry about designing a reward function.


A New AI Study May Explain Why Deep Learning Works

#artificialintelligence

The resurgence of artificial intelligence (AI) is largely due to advances in pattern-recognition due to deep learning, a form of machine learning that does not require explicit hard-coding. The architecture of deep neural networks is somewhat inspired by the biological brain and neuroscience. Like the biological brain, the inner workings of exactly why deep networks work are largely unexplained, and there is no single unifying theory. Recently researchers at the Massachusetts Institute of Technology (MIT) revealed new insights about how deep learning networks work to help further demystify the black box of AI machine learning. The MIT research trio of Tomaso Poggio, Andrzej Banburski, and Quianli Liao at the Center for Brains, Minds, and Machines developed a new theory as to why deep networks work and published their study published on June 9, 2020 in PNAS (Proceedings of the National Academy of Sciences of the United States of America).


Move over neural networks! – A new method for cosmological inference

#artificialintelligence

Neural networks have seen a lot of hype in astronomy and cosmology recently (even just on this site! However, it may be that the neural networks used to classify images in typical machine learning applications are overkill. To quote the authors of today's paper, "the cosmological density field is not as complex as random images of rabbits." Today's authors propose using a method called the "scattering transform" to take advantage of the best parts of neural networks with none of the limitations. The standard cosmological lore states that early on the universe underwent a phase of inflation that is responsible for laying down the initial conditions of the large-scale structure we see in the universe today.


A visual explanation for regularization of linear models

#artificialintelligence

Personally, my biggest initial stumbling block was this: The math used to implement regularization does not correspond to pictures commonly used to explain regularization. Take a look at the oft-copied picture (shown below left) from page 71 of ESL in the section on "Shrinkage Methods." Students see this multiple times in their careers but have trouble mapping that to the relatively straightforward mathematics used to regularize linear model training. The simple reason is that that illustration shows how we regularize models conceptually, with hard constraints, not how we actually implement regularization, with soft constraints! Regularization conceptually uses a hard constraint to prevent coefficients from getting too large (the cyan circles from the ESL picture).


Looking into the black box of deep learning - ScienceBlog.com

#artificialintelligence

Deep learning systems are revolutionizing technology around us, from voice recognition that pairs you with your phone to autonomous vehicles that are increasingly able to see and recognize obstacles ahead. But much of this success involves trial and error when it comes to the deep learning networks themselves. A group of MIT researchers recently reviewed their contributions to a better theoretical understanding of deep learning networks, providing direction for the field moving forward. "Deep learning was in some ways an accidental discovery," explains Tommy Poggio, investigator at the McGovern Institute for Brain Research, director of the Center for Brains, Minds, and Machines (CBMM), and the Eugene McDermott Professor in Brain and Cognitive Sciences. "We still do not understand why it works. A theoretical framework is taking form, and I believe that we are now close to a satisfactory theory. It is time to stand back and review recent insights."


Looking into the black box

#artificialintelligence

Deep learning systems are revolutionizing technology around us, from voice recognition that pairs you with your phone to autonomous vehicles that are increasingly able to see and recognize obstacles ahead. But much of this success involves trial and error when it comes to the deep learning networks themselves. A group of MIT researchers recently reviewed their contributions to a better theoretical understanding of deep learning networks, providing direction for the field moving forward. "Deep learning was in some ways an accidental discovery," explains Tommy Poggio, investigator at the McGovern Institute for Brain Research, director of the Center for Brains, Minds, and Machines (CBMM), and the Eugene McDermott Professor in Brain and Cognitive Sciences. "We still do not understand why it works. A theoretical framework is taking form, and I believe that we are now close to a satisfactory theory. It is time to stand back and review recent insights."


Seeing Light at the End of the Cybersecurity Tunnel

Communications of the ACM

ACM athena award recipient Elisa Bertino, a professor at Purdue University and research director of the Cyber Space Security Lab of Purdue's Department of Computer Science, has spent her career trying to ensure the security and integrity of the information that is stored in databases and transmitted over mobile, social, cloud, Internet of Things (IoT), and sensor networks. Here, she talks about how her research interests have evolved and why she's not pessimistic about the future of cybersecurity. You began your research career in the field of databases, first at the Italian National Research Council, and later as a post-doc at IBM's San Jose Research Laboratory. What drew you to security? My original interest in security began at IBM, where I was looking into how to protect the data stored in databases.


Bayesian Inference: The Maximum Entropy Principle

#artificialintelligence

In this article, I will explain what the maximum entropy principle is, how to apply it and why it's useful in the context of Bayesian inference. The code to reproduce the results and figures can be found in this notebook. The maximum entropy principle is a method to create probability distributions that is most consistent with a given set of assumptions and nothing more. The rest of the article will explain what this means. First, we need to a way to measure the uncertainty in a probability distribution.