Education
Failure Modes in Machine Learning Systems
Kumar, Ram Shankar Siva, Brien, David O, Albert, Kendra, Viljöen, Salomé, Snover, Jeffrey
In the last two years, more than 200 papers have been written on how machine learning (ML) systems can fail because of adversarial attacks on the algorithms and data; this number balloons if we were to incorporate papers covering non-adversarial failure modes. The spate of papers has made it difficult for ML practitioners, let alone engineers, lawyers, and policymakers, to keep up with the attacks against and defenses of ML systems. However, as these systems become more pervasive, the need to understand how they fail, whether by the hand of an adversary or due to the inherent design of a system, will only become more pressing. In order to equip software developers, security incident responders, lawyers, and policy makers with a common vernacular to talk about this problem, we developed a framework to classify failures into "Intentional failures" where the failure is caused by an active adversary attempting to subvert the system to attain her goals; and "Unintentional failures" where the failure is because an ML system produces an inherently unsafe outcome. After developing the initial version of the taxonomy last year, we worked with security and ML teams across Microsoft, 23 external partners, standards organization, and governments to understand how stakeholders would use our framework. Throughout the paper, we attempt to highlight how machine learning failure modes are meaningfully different from traditional software failures from a technology and policy perspective.
Corpus Wide Argument Mining -- a Working Solution
Ein-Dor, Liat, Shnarch, Eyal, Dankin, Lena, Halfon, Alon, Sznajder, Benjamin, Gera, Ariel, Alzate, Carlos, Gleize, Martin, Choshen, Leshem, Hou, Yufang, Bilu, Yonatan, Aharonov, Ranit, Slonim, Noam
One of the main tasks in argument mining is the retrieval of argumentative content pertaining to a given topic. Most previous work addressed this task by retrieving a relatively small number of relevant documents as the initial source for such content. This line of research yielded moderate success, which is of limited use in a real-world system. Furthermore, for such a system to yield a comprehensive set of relevant arguments, over a wide range of topics, it requires leveraging a large and diverse corpus in an appropriate manner. Here we present a first end-to-end high-precision, corpus-wide argument mining system. This is made possible by combining sentence-level queries over an appropriate indexing of a very large corpus of newspaper articles, with an iterative annotation scheme. This scheme addresses the inherent label bias in the data and pinpoints the regions of the sample space whose manual labeling is required to obtain high-precision among top-ranked candidates. 1 Introduction Starting with the seminal work of Mochales Palau and Moens (2009), argument mining has mainly focused on the following tasks - identifying argumentative text segments within a given document; labeling these text segments according to the type of argument and its stance; and elucidating the discourse relations among the detected arguments. Typically, the considered documents were argumentative in nature, taken from a well defined domain, such as legal documents or student essays. More recently, some attention had been given to the corresponding retrieval task - given a controversial topic, retrieve arguments with a clear stance towards this topic. This is usually done by first retrieving - manually or automatically - documents relevant to the topic, and then using argument mining techniques to identify relevant argumentative segments therein. This documents-based approach was originally explored over Wikipedia (Levy et al. 2014; Rinott et al. 2015), and more recently over the entire Web (Stab et al. 2018). For an argument retrieval system to be of practical use requires: (1) high precision, and (2) wide coverage.
On the Measure of Intelligence
To make deliberate progress towards more intelligent and more human-like artificial systems, we need to be following an appropriate feedback signal: we need to be able to define and evaluate intelligence in a way that enables comparisons between two systems, as well as comparisons with humans. Over the past hundred years, there has been an abundance of attempts to define and measure intelligence, across both the fields of psychology and AI. We summarize and critically assess these definitions and evaluation approaches, while making apparent the two historical conceptions of intelligence that have implicitly guided them. We note that in practice, the contemporary AI community still gravitates towards benchmarking intelligence by comparing the skill exhibited by AIs and humans at specific tasks such as board games and video games. We argue that solely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experience: unlimited priors or unlimited training data allow experimenters to "buy" arbitrary levels of skills for a system, in a way that masks the system's own generalization power. We then articulate a new formal definition of intelligence based on Algorithmic Information Theory, describing intelligence as skill-acquisition efficiency and highlighting the concepts of scope, generalization difficulty, priors, and experience. Using this definition, we propose a set of guidelines for what a general AI benchmark should look like. Finally, we present a benchmark closely following these guidelines, the Abstraction and Reasoning Corpus (ARC), built upon an explicit set of priors designed to be as close as possible to innate human priors. We argue that ARC can be used to measure a human-like form of general fluid intelligence and that it enables fair general intelligence comparisons between AI systems and humans.
What's New in EDU: Introducing a new Minecraft Hour of Code tutorial with AI and the Discovery STEM Careers Coalition Microsoft EDU
There's a good chance the students you're teaching today will enroll in university courses that haven't yet been created and enter jobs that don't exist. And they'll be called upon to solve some of the world's most pressing environmental, social and economic issues. We know that can feel like a lot on your shoulders, but there is plenty you can do to prepare students for success and we're here to help. Thoughtfully designed and well implemented STEM instruction builds subject-specific knowledge and fosters a growth mindset, collaboration, critical thinking and computational thinking – all vital skills for jobs of the future. We have tips to share about fun ways to participate in Hour of Code, available in Minecraft: Education Edition as a free coding lesson.
4 reasons why HR leaders need to be ready for AI in their workplaces
One of the biggest change agents in organisations today is technology. Technology has changed and continues to change the way organisations are structured, through the likes of virtual teams, remote workplaces and outsourcing of departments to other countries. Ways in which organisations work are also changing, through the likes of online meeting rooms; e-learning, online performance appraisals, video recruiting, virtual customer service platforms, chatbots and e-financial platforms. There are a range of technologies disrupting organisations and HR, but the one that can be considered the biggest is AI. AI in the form of chatbots, learning platforms, machine learning systems, voice recognition systems as well as virtual assistants, are becoming more commonplace in the workplace.
MIT conference focuses on preparing workers for the era of artificial
In opening yesterday's AI and the Work of the Future Congress, MIT Professor Daniela Rus presented diverging views of how artificial intelligence will impact jobs worldwide. By automating certain menial tasks, experts think AI is poised to improve human quality of life, boost profits, and create jobs, said Rus, director of the Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science. Rus then quoted a World Economic Forum study estimating AI could help create 133 million new jobs worldwide over the next five years. Juxtaposing this optimistic view, however, she noted a recent survey that found about two-thirds of Americans believe machines will soon rob humans of their careers. The economists, who predict greater productivity and new jobs?
Prosthetic legs of California high school wrestling captain stolen from gym
Fox News Flash top headlines for Nov. 24 are here. Check out what's clicking on FoxNews.com The prosthetic legs of a double amputee and soon-to-be high school wrestling captain were stolen from a gym closet in California last week, putting his dreams of winning a state championship or even wrestling this season in doubt. Brett Winters, a senior at Pacific High School in San Bernardino, California, was born without tibia bones in his legs. As a baby, his mother was told by doctors that Winters could either spend life in a wheelchair or amputate his legs.
To secure a safer future for AI, we need the benefit of a female perspective John Naughton
Everybody knows (or should know) by now that machine learning (which is what most current artificial intelligence actually amounts to) is subject to bias. Last week, the New York Times had the idea of asking three prominent experts in the field to talk about the bias problem, in particular the ways that social bias can be reflected and amplified in dangerous ways by the technology to discriminate against, or otherwise damage, certain social groups. At first sight, the resulting article looked like a run-of-the-mill review of what has become a common topic – except for one thing: the three experts were all women. One, Daphne Koller, is a co-founder of the online education company Coursera; another, Olga Russakovsky, is a Princeton professor who is working to reduce bias in ImageNet, the data set that powered the current machine-learning boom; the third, Timnit Gebru, is a research scientist at Google in the company's ethical AI team. Reading the observations of these three women brought to the surface a thought that's been lurking at the back of my mind for years.
ASSISTANT TEACHING PROFESSOR-Lecturer with Potential Security of Employment in Machine Learning
We invite applications for a Tenure Track Assistant Teaching Professor position in Machine Learning. We interpret this area broadly and invite candidates who can provide students with strong foundations in machine learning, deep learning, neural networks, and/or visual computation. We are especially interested in candidates who will flourish in a Cognitive Science Department, and whose research, teaching, or service has prepared them to contribute to our commitment to diversity, inclusion, and equity within an academic setting. Joint appointment with other departments can be considered where appropriate. The Assistant Teaching Professor is also known within the UC as an LPSOE (Lecturer with Potential for Security of Employment).
DesignCon Expands Into Artificial Intelligence, Automotive, 5G, IoT, and More For 2020 Edition
DesignCon, the nation's largest event for chip, board, and systems design engineers, today announced new areas of focus for the 2020 edition highlighting advances in the fields of 5G, artificial intelligence (AI), automotive, and IoT, producing the most in-demand electronics emerging today. Topics will be examined through a 14-track conference schedule spanning technical sessions, boot camps, tutorials, and more to fit the needs of the hardware design engineering community. The DesignCon conference and expo reflects the design engineering industry's growing need for breakthroughs in the development of 5G connectivity, AI, automotive electronics, and IoT. With worldwide spending on AI systems forecasted to reach $79.2 billion in 2022, sales of automotive electronic systems at its highest growth rate on-record, expected worldwide spending on IoT projected to surpass $1 trillion in 2022, and the 5G infrastructure market estimated to reach $4.2 billion in value by 2020, DesignCon is the ideal forum for attendees across these interlaced industries to exchange ideas and develop solutions to meet consumer demand. "The market and value for connected, especially fast connected products, is on the rise," said Suzanne Deffree, brand director, Intelligent Systems & Design, Informa Markets.