Education
Teaching AI, Ethics, Law and Policy
The cyberspace and the development of new technologies, especially intelligent systems using artificial intelligence, present enormous challenges to computer professionals, data scientists, managers and policy makers. There is a need to address professional responsibility, ethical, legal, societal, and policy issues. This paper presents problems and issues relevant to computer professionals and decision makers and suggests a curriculum for a course on ethics, law and policy. Such a course will create awareness of the ethics issues involved in building and using software and artificial intelligence.
Introduction to Multi-Armed Bandits
Multi-armed bandits a simple but very powerful framework for algorithms that make decisions over time under uncertainty. An enormous body of work has accumulated over the years, covered in several books and surveys. This book provides a more introductory, textbook-like treatment of the subject. Each chapter tackles a particular line of work, providing a self-contained, teachable technical introduction and a review of the more advanced results. The chapters are as follows: Stochastic bandits; Lower bounds; Bayesian Bandits and Thompson Sampling; Lipschitz Bandits; Full Feedback and Adversarial Costs; Adversarial Bandits; Linear Costs and Semi-bandits; Contextual Bandits; Bandits and Zero-Sum Games; Bandits with Knapsacks; Incentivized Exploration and Connections to Mechanism Design.
Will Artificial Intelligence Enhance or Hack Humanity?
This week, I interviewed Yuval Noah Harari, the author of three best-selling books about the history and future of our species, and Fei-Fei Li, one of the pioneers in the field of artificial intelligence. The event was hosted by the Stanford Center for Ethics and Society, the Stanford Institute for Human-Centered Artificial Intelligence, and the Stanford Humanities Center. A transcript of the event follows, and a video is posted below. Nicholas Thompson: Thank you, Stanford, for inviting us all here. I want this conversation to have three parts: First, lay out where we are; then talk about some of the choices we have to make now; and last, talk about some advice for all the wonderful people in the hall. Yuval, the last time we talked, you said many, many brilliant things, but one that stuck out was a line where you said, "We are not just in a technological crisis. We are in a philosophical crisis." So explain what you meant and explain how it ties to AI. Let's get going with a note of ...
Conditional Teacher-Student Learning
Meng, Zhong, Li, Jinyu, Zhao, Yong, Gong, Yifan
The teacher-student (T/S) learning has been shown to be effective for a variety of problems such as domain adaptation and model compression. One shortcoming of the T/S learning is that a teacher model, not always perfect, sporadically produces wrong guidance in form of posterior probabilities that misleads the student model towards a suboptimal performance. To overcome this problem, we propose a conditional T/S learning scheme, in which a "smart" student model selectively chooses to learn from either the teacher model or the ground truth labels conditioned on whether the teacher can correctly predict the ground truth. Unlike a naive linear combination of the two knowledge sources, the conditional learning is exclusively engaged with the teacher model when the teacher model's prediction is correct, and otherwise backs off to the ground truth. Thus, the student model is able to learn effectively from the teacher and even potentially surpass the teacher. We examine the proposed learning scheme on two tasks: domain adaptation on CHiME-3 dataset and speaker adaptation on Microsoft short message dictation dataset. The proposed method achieves 9.8% and 12.8% relative word error rate reductions, respectively, over T/S learning for environment adaptation and speaker-independent model for speaker adaptation.
Deep pNML: Predictive Normalized Maximum Likelihood for Deep Neural Networks
Bibas, Koby, Fogel, Yaniv, Feder, Meir
The Predictive Normalized Maximum Likelihood (pNML) scheme has been recently suggested for universal learning in the individual setting, where both the training and test samples are individual data. The goal of universal learning is to compete with a ``genie'' or reference learner that knows the data values, but is restricted to use a learner from a given model class. The pNML minimizes the associated regret for any possible value of the unknown label. Furthermore, its min-max regret can serve as a pointwise measure of learnability for the specific training and data sample. In this work we examine the pNML and its associated learnability measure for the Deep Neural Network (DNN) model class. As shown, the pNML outperforms the commonly used Empirical Risk Minimization (ERM) approach and provides robustness against adversarial attacks. Together with its learnability measure it can detect out of distribution test examples, be tolerant to noisy labels and serve as a confidence measure for the ERM. Finally, we extend the pNML to a ``twice universal'' solution, that provides universality for model class selection and generates a learner competing with the best one from all model classes.
Machine Learning in the Air
Gunduz, Deniz, de Kerret, Paul, Sidiropoulos, Nicholas D., Gesbert, David, Murthy, Chandra, van der Schaar, Mihaela
Thanks to the recent advances in processing speed and data acquisition and storage, machine learning (ML) is penetrating every facet of our lives, and transforming research in many areas in a fundamental manner. Wireless communications is another success story -- ubiquitous in our lives, from handheld devices to wearables, smart homes, and automobiles. While recent years have seen a flurry of research activity in exploiting ML tools for various wireless communication problems, the impact of these techniques in practical communication systems and standards is yet to be seen. In this paper, we review some of the major promises and challenges of ML in wireless communication systems, focusing mainly on the physical layer. We present some of the most striking recent accomplishments that ML techniques have achieved with respect to classical approaches, and point to promising research directions where ML is likely to make the biggest impact in the near future. We also highlight the complementary problem of designing physical layer techniques to enable distributed ML at the wireless network edge, which further emphasizes the need to understand and connect ML with fundamental concepts in wireless communications.
Immigration Services Agency to toughen Japanese-language school standards
The Immigration Services Agency plans to strengthen its eligibility standards for Japanese-language schools, it was learned Saturday. The move comes as Japanese-language schools have been under fire for accepting many foreign students whose purpose is to work in Japan. The number of Japanese-language schools recognized by the government grew 1.6 times over the past five years to 749 as of April 2. The government late last year outlined plans to improve the quality of Japanese-language schools as part of efforts to bring in more foreign workers to the country. Under the agency's plan, the requirement for the average student attendance rate would be revised from the current 50 percent or more in a month to 70 percent or more in a period of seven months. Schools failing to meet the requirement would not be allowed to accept foreign students.
Headspace: How the meditation app turns your stressful phone into a source of calm
Meditation and mindfulness have been around for thousands of years. But the advent of smartphones and computers led to a new phenomenon: the mindfulness app. There are a few to choose from, including the punchy, assertive 10% Happier, the elegant and placid Calm and the first app that really brought mindfulness to our phones, Headspace. Andy Puddicombe, a former Buddhist monk who went on to run a meditation clinic in London, met a new business partner, Richard Pierson, and launched Headspace in 2010. The company began as an events organisation and led to the now-ubiquitous app in 2012.
Three ways to build a strong AI-training pipeline
Artificial-intelligence researcher Oren Etzioni has suggestions for keeping enough AI faculty members around to train the next generation.Credit: Bret Hartman/TED Oren Etzioni is chief executive of the non-profit Allen Institute for Artificial Intelligence (AI2) in Seattle, Washington, and is on leave from the nearby University of Washington. He offers some recommendations for how to stem the outflow of artificial-intelligence (AI) researchers from academia to industry -- a loss that is damaging academia's ability to teach incoming undergraduates. It is a very sizeable trend for fresh PhD graduates and faculty members. In machine learning, you see some significant departures. Industry compensation packages are highly variable.
Prediction with Unpredictable Feature Evolution
Hou, Bo-Jian, Zhang, Lijun, Zhou, Zhi-Hua
Feature space can change or evolve when learning with streaming data. Several recent works have studied feature evolvable learning. They usually assume that features would not vanish or appear in an arbitrary way. For example, when knowing the battery lifespan, old features and new features represented by data gathered by sensors will disappear and emerge at the same time along with the sensors exchanging simultaneously. However, different sensors would have different lifespans, and thus the feature evolution can be unpredictable. In this paper, we propose a novel paradigm: Prediction with Unpredictable Feature Evolution (PUFE). We first complete the unpredictable overlapping period into an organized matrix and give a theoretical bound on the least number of observed entries. Then we learn the mapping from the completed matrix to recover the data from old feature space when observing the data from new feature space. With predictions on the recovered data, our model can make use of the advantage of old feature space and is always comparable with any combinations of the predictions on the current instance.