Deep convolutional neural networks (CNNs) with a large number of parameters requires huge computational resources, which has limited the application of CNNs on resources constrained appliances. Decomposition-based methods, therefore, have been utilized to compress CNNs in recent years. However, since the compression factor and performance are negatively correlated, the state-of-the-art works either suffer from severe performance degradation or have limited low compression factors. To overcome these problems, unlike previous works compressing layers separately, we propose to compress CNNs and alleviate performance degradation via joint matrix decomposition. The idea is inspired by the fact that there are lots of repeated modules in CNNs, and by projecting weights with the same structures into the same subspace, networks can be further compressed and even accelerated. In particular, three joint matrix decomposition schemes are developed, and the corresponding optimization approaches based on Singular Values Decomposition are proposed. Extensive experiments are conducted across three challenging compact CNNs and 3 benchmark data sets to demonstrate the superior performance of our proposed algorithms. As a result, our methods can compress the size of ResNet-34 by 22x with slighter accuracy degradation compared with several state-of-the-art methods.

How to Learn Machine Learning – Tips and Resources to Learn ML the Practical Way Yacine Mahdid A lot of people want to learn machine learning these days. But the daunting bottom-up curriculum that most ML teachers propose is enough discourage a lot of newcomers. In this tutorial I flip the curriculum upside down and will outline what I think is the fastest and easiest way to get a solid grasp of ML. Table of Contents Step 6: Repeat steps 0 to 5 This is a looping learning plan because the 6th step is actually a GOTO to Step 0! As a disclaimer, this curriculum might strange to you. But I've battle tested it when I was teaching machine learning to undergraduates at McGill University. I tried many iteration of this curriculum, starting with the theoretically superior bottom-up approach. But from experience, this pragmatic top-down approach is what gives the best results. One common critique I get is that people not starting with the basics, like statistics or linear algebra, will have a poor understanding of machine learning and they will not know what they are doing when modeling. In theory, yes, this is true and this is why I started teaching ML with the bottom up approach. In practice, this has never been the case. What actually ended up happening was that because the students knew how to do the high level modeling, they were much more inclined to delve into the low level stuff on their own as they saw the direct benefit it would bring to their higher level skills. This context that they were able to set for themselves wouldn't have been there if they'd started from the bottom – and this is where I believe most teachers lose their students. All that being said, let's jump into the actual learning plan!

GPT-3 IS QUITE a beast. The Generative Pre-Trained Transformer 3, to give its full name, is a language model developed by OpenAI, a part-commercial, part not-for-profit artificial-intelligence (AI) laboratory in San Francisco. GPT-3 was trained on an unprecedented mass of text to teach it the probability that a given word will follow preceding words. When fed a short text "prompt", it cranks out astonishingly coherent prose written in a similar style. Your browser does not support the audio element.

Artificial intelligence, machine learning, deep learning, blockchain … But the question is that if artificial intelligence and robots replace humans in …

This post was written for TechTalks by Herbert Roitblat, the author of Algorithms Are Not Enough: How to Create Artificial General Intelligence. In a recent paper, the DeepMind team, (Silver et al., 2021) argue that rewards are enough for all kinds of intelligence. Specifically, they argue that "maximizing reward is enough to drive behavior that exhibits most if not all attributes of intelligence." They argue that simple rewards are all that is needed for agents in rich environments to develop multi-attribute intelligence of the sort needed to achieve artificial general intelligence. This sounds like a bold claim, but, in fact, it is so vague as to be almost meaningless. They support their thesis, not by offering specific evidence, but by repeatedly asserting that reward is enough because the observed solutions to the problems are consistent with the problem having been solved.

My latest project in using AI for creative endeavors is called MAGnet. I built a custom Genetic Algorithm (GA) to drive the creation of modern paintings using a Generative Adversarial Network (GAN) from a textual description over several generations. MAGnet uses the CLIP model from OpenAI [1] and a variant of StyleGAN2 ADA [2] from Nvidia called SWAGAN [3], which uses wavelets to create images. All of the source code for this project is available here. A Google Colab that you can use to create your own paintings is available here.

Generative Adversarial Networks (GANs), proposed by Goodfellow et al. in 2014, revolutionized a domain of image generation in computer vision -- no one could believe that these stunning and lively images are actually generated purely by machines. In fact, people used to think the task of generation was impossible and were surprised with the power of GAN, because traditionally, there simply is no ground truth we can compare our generated images to.

Sentient Vision Systems is an artificial intelligence (AI) company that uses advanced software to enhance the performance of sensors and mission systems. ViDAR (for Visual Detection and Ranging) can detect a target in the imagery feed, discriminate between possible alternatives, and draw the operator's eye to what he or she is looking for. The power of AI can differentiate, from a distance of five nautical miles, between an arctic ice floe, a breaking wave and an upturned boat. AI and mastery of traditional computer vision technology underpins everything that Sentient Vision Systems has done over the past 17 years, since it started working on target detection solutions over land and maritime environments. Sentient's ViDAR systems use the AI within its deep learning and computer vision algorithms to detect tiny targets that are almost invisible in the imagery feed from an EO/IR sensor, especially in very challenging conditions, and filter out irrelevant information.

Deep neural networks will move past their shortcomings without help from symbolic artificial intelligence, three pioneers of deep learning argue in a paper published in the July issue of the Communications of the ACM journal. In their paper, Yoshua Bengio, Geoffrey Hinton, and Yann LeCun, recipients of the 2018 Turing Award, explain the current challenges of deep learning and how it differs from learning in humans and animals. They also explore recent advances in the field that might provide blueprints for the future directions for research in deep learning. Titled "Deep Learning for AI," the paper envisions a future in which deep learning models can learn with little or no help from humans, are flexible to changes in their environment, and can solve a wide range of reflexive and cognitive problems. Above: Deep learning pioneers Yoshua Bengio (left), Geoffrey Hinton (center), and Yann LeCun (right).

As deep learning techniques continue to advance, image recognition systems are becoming more and more powerful. With this power comes great reward -- helping diagnose disease from x-rays and self-driving cars are just two examples. But there is also potential for harm, particularly concerning facial recognition. In the future, it's possible that surveillance cameras with state-of-the-art facial recognition technology could pop up on every street corner, effectively eliminating any privacy we still have. Fortunately, some researchers are already coming up with ways to counteract deep learning based facial recognition. I would like to highlight one interesting method -- using an adversarial attack in the form of specially colored glasses to confuse facial recognition algorithms.