The Mean Teacher (MT) model of Tarvainen and Valpola has shown favorable performance on several semi-supervised benchmark datasets. MT maintains a teacher model's weights as the exponential moving average of a student model's weights and minimizes the divergence between their probability predictions under diverse perturbations of the inputs. However, MT is known to suffer from confirmation bias, that is, reinforcing incorrect teacher model predictions. In this work, we propose a simple yet effective method called Local Clustering (LC) to mitigate the effect of confirmation bias. In MT, each data point is considered independent of other points during training; however, data points are likely to be close to each other in feature space if they share similar features. Motivated by this, we cluster data points locally by minimizing the pairwise distance between neighboring data points in feature space. Combined with a standard classification cross-entropy objective on labeled data points, the misclassified unlabeled data points are pulled towards high-density regions of their correct class with the help of their neighbors, thus improving model performance. We demonstrate on semi-supervised benchmark datasets SVHN and CIFAR-10 that adding our LC loss to MT yields significant improvements compared to MT and performance comparable to the state of the art in semi-supervised learning.

Networks of spiking neurons and Winner-Take-All spiking circuits (WTA-SNNs) can detect information encoded in spatio-temporal multi-valued events. These are described by the timing of events of interest, e.g., clicks, as well as by categorical numerical values assigned to each event, e.g., like or dislike. Other use cases include object recognition from data collected by neuromorphic cameras, which produce, for each pixel, signed bits at the times of sufficiently large brightness variations. Existing schemes for training WTA-SNNs are limited to rate-encoding solutions, and are hence able to detect only spatial patterns. Developing more general training algorithms for arbitrary WTA-SNNs inherits the challenges of training (binary) Spiking Neural Networks (SNNs). These amount, most notably, to the non-differentiability of threshold functions, to the recurrent behavior of spiking neural models, and to the difficulty of implementing backpropagation in neuromorphic hardware. In this paper, we develop a variational online local training rule for WTA-SNNs, referred to as VOWEL, that leverages only local pre- and post-synaptic information for visible circuits, and an additional common reward signal for hidden circuits. The method is based on probabilistic generalized linear neural models, control variates, and variational regularization. Experimental results on real-world neuromorphic datasets with multi-valued events demonstrate the advantages of WTA-SNNs over conventional binary SNNs trained with state-of-the-art methods, especially in the presence of limited computing resources.

Edge Computing runs processes locally on the device itself, instead of running them in the cloud. This reduced computing time allows data to be processed much faster, removes the security risk of transferring the data to a cloud-based server, and reduces the cost of data transfer, as well as the risks of bandwidth outages disrupting performance. Computer vision and AI at the edge are becoming instrumental in powering everything from factory assembly lines and retail inventory management, to hospital urgent care medical imaging equipment like X-ray and CAT scans. Drones, security cameras, robots, facial recognition on cell phones, self-driving vehicles, and more all utilize this technology as well. According to IEEE Innovation at Work, "By 2020, approximately 20 billion devices will likely be connected via the Internet of Things (IoT), creating incredible amounts of data every minute. The time it takes to move data to the cloud, perform service on it and then move it back to devices is far too long to meet the increasing needs of the IoT. Unlike cloud computing, which relies on a single data center, edge computing works with a more distributed network, eliminating the round-trip journey to the cloud and offering real-time responsiveness and local authority. It keeps the heaviest traffic and processing closest to the end-user application and devices – smartphones, tablets, home security systems, and more – that generate and consume data. This dramatically reduces latency and leads to real-time, automated decision-making."

For this week's ML practitioner's series, Analytics India Magazine got in touch with Arthur Llau. Arthur is a Kaggle master, who is currently ranked in the top 100 on the global leaderboard that hosts more than 1,30,000 participants. He is a mathematician from heart, who happened to run into machine learning. We bring to our readers Arthur Llau's fascinating journey into the world of data science. A lifelong Parisian, Arthur Llau has a dual masters degree in Theoretical Mathematics (Probability) and in Statistics & Machine Learning from Sorbonne University campus of Université Pierre & Marie Curie.

Before the coronavirus hit, consumer expectations were already changing and creating challenges in the retail industry. And CIOs, who historically had little to do with developing new customer experiences, were increasingly being tasked with driving innovation. To deliver Amazon-like here-and-now products and services and build brand loyalty, they are turning to artificial intelligence (AI) to improve the shopping experience for consumers both in stores and online. Of course, AI alone won't transform retail but there are several key technologies that when married with AI can bring innovation to that industry. One is video analytics, which turns regular stores into intelligent stores that have visibility into consumer behavior for optimized merchandising.

Knowledge Distillation (KD) is a common method for transferring the ``knowledge'' learned by one machine learning model (the \textit{teacher}) into another model (the \textit{student}), where typically, the teacher has a greater capacity (e.g., more parameters or higher bit-widths). To our knowledge, existing methods overlook the fact that although the student absorbs extra knowledge from the teacher, both models share the same input data -- and this data is the only medium by which the teacher's knowledge can be demonstrated. Due to the difference in model capacities, the student may not benefit fully from the same data points on which the teacher is trained. On the other hand, a human teacher may demonstrate a piece of knowledge with individualized examples adapted to a particular student, for instance, in terms of her cultural background and interests. Inspired by this behavior, we design data augmentation agents with distinct roles to facilitate knowledge distillation. Our data augmentation agents generate distinct training data for the teacher and student, respectively. We find empirically that specially tailored data points enable the teacher's knowledge to be demonstrated more effectively to the student. We compare our approach with existing KD methods on training popular neural architectures and demonstrate that role-wise data augmentation improves the effectiveness of KD over strong prior approaches. The code for reproducing our results can be found at https://github.com/bigaidream-projects/role-kd

In light of the recent events surrounding COVID-19, in order to make it easier for K-12 students to learn and be productive, a team led by Media Lab Associate Professor Cynthia Breazeal has launched a website to share a variety of online activities to learn about artificial intelligence. Due to this COVID-19 pandemic, parents and educators must be feeling overwhelmed in providing K-12 students classroom level education, and that's why the team decided to create this website consisting of MIT resources to learn about designing artificial intelligence and its responsible uses. The website has been created in collaboration between the Media Lab, MIT Stephen A. Schwarzman College of Computing, and MIT Open Learning. It acts as a hub to highlight diverse work by faculty, staff, and students across the MIT community at the intersection of AI, learning, and education. According to the team, it will be interesting for K-12 students to learn from this website.

One of the most common complaints I hear from students is: Why do I have to learn all this math? Why isn't there a library to do what I want? Someone recently made this proclamation to me: "You should explain that your courses are for college students, not industry professionals". This made me laugh very hard. In this article, I will refer to students who make such proclamations as "ML wannabes" for lack of a better term, because people who actually do ML generally know better than this.

During the COVID–19 pandemic, I decided that I wanted to use the time at home to invest in myself. One of the things I was challenged by in a recent episode of Business Casual was when Mark Cuban discussed the role of Artificial Intelligence in the future and recommended some tools to learn more. He mentioned some Coursera courses, so I am currently working my way through some of their AI training, but he also mentioned an AI-focused podcast called the TWIML AI Podcast that I added to my podcast subscription list. TWIML (This Week in Machine Learning and AI) is a perfect way to hear from industry experts about how Machine Learning and AI will change our world. I plan to work through the back catalog soon, but the newest episodes have been informative.

Retailers looking to develop new customer experiences need to make artificial intelligence part of their digital transformation plans or risk falling further behind. Before the coronavirus hit, consumer expectations were already changing and creating challenges in the retail industry. And CIOs, who historically had little to do with developing new customer experiences, were increasingly being tasked with driving innovation. To deliver Amazon-like here-and-now products and services and build brand loyalty, they are turning to artificial intelligence (AI) to improve the shopping experience for consumers both in stores and online. Of course, AI alone won't transform retail but there are several key technologies that when married with AI can bring innovation to that industry.