Goto

Collaborating Authors

 Education


FairyTED: A Fair Rating Predictor for TED Talk Data

arXiv.org Machine Learning

With the recent trend of applying machine learning in every aspect of human life, it is important to incorporate fairness into the core of the predictive algorithms. We address the problem of predicting the quality of public speeches while being fair with respect to sensitive attributes of the speakers, e.g. gender and race. We use the TED talks as an input repository of public speeches because it consists of speakers from a diverse community and has a wide outreach. Utilizing the theories of Causal Models, Counterfactual Fairness and state-of-the-art neural language models, we propose a mathematical framework for fair prediction of the public speaking quality. We employ grounded assumptions to construct a causal model capturing how different attributes affect public speaking quality. This causal model contributes in generating counterfactual data to train a fair predictive model. Our framework is general enough to utilize any assumption within the causal model. Experimental results show that while prediction accuracy is comparable to recent work on this dataset, our predictions are counterfactually fair with respect to a novel metric when compared to true data labels. The FairyTED setup not only allows organizers to make informed and diverse selection of speakers from the unobserved counterfactual possibilities but it also ensures that viewers and new users are not influenced by unfair and unbalanced ratings from arbitrary visitors to the www.ted.com website when deciding to view a talk.


Cumulative Sum Ranking

arXiv.org Machine Learning

The goal of Ordinal Regression is to find a rule that ranks items from a given set. Several learning algorithms to solve this prediction problem build an ensemble of binary classifiers. Ranking by Projecting uses interdependent binary perceptrons. These perceptrons share the same direction vector, but use different bias values. Similar approaches use independent direction vectors and biases. To combine the binary predictions, most of them adopt a simple counting heuristics. Here, we introduce a novel cumulative sum scoring function to combine the binary predictions. The proposed score value aggregates the strength of each one of the relevant binary classifications on how large is the item's rank. We show that our modeling casts ordinal regression as a Structured Perceptron problem. As a consequence, we simplify its formulation and description, which results in two simple online learning algorithms. The second algorithm is a Passive-Aggressive version of the first algorithm. We show that under some rank separability condition both algorithms converge. Furthermore, we provide mistake bounds for each one of the two online algorithms. For the Passive-Aggressive version, we assume the knowledge of a separation margin, what significantly improves the corresponding mistake bound. Additionally, we show that Ranking by Projecting is a special case of our prediction algorithm. From a neural network architecture point of view, our empirical findings suggest a layer of cusum units for ordinal regression, instead of the usual softmax layer of multiclass problems.


Failure Modes in Machine Learning Systems

arXiv.org Machine Learning

In the last two years, more than 200 papers have been written on how machine learning (ML) systems can fail because of adversarial attacks on the algorithms and data; this number balloons if we were to incorporate papers covering non-adversarial failure modes. The spate of papers has made it difficult for ML practitioners, let alone engineers, lawyers, and policymakers, to keep up with the attacks against and defenses of ML systems. However, as these systems become more pervasive, the need to understand how they fail, whether by the hand of an adversary or due to the inherent design of a system, will only become more pressing. In order to equip software developers, security incident responders, lawyers, and policy makers with a common vernacular to talk about this problem, we developed a framework to classify failures into "Intentional failures" where the failure is caused by an active adversary attempting to subvert the system to attain her goals; and "Unintentional failures" where the failure is because an ML system produces an inherently unsafe outcome. After developing the initial version of the taxonomy last year, we worked with security and ML teams across Microsoft, 23 external partners, standards organization, and governments to understand how stakeholders would use our framework. Throughout the paper, we attempt to highlight how machine learning failure modes are meaningfully different from traditional software failures from a technology and policy perspective.


Corpus Wide Argument Mining -- a Working Solution

arXiv.org Artificial Intelligence

One of the main tasks in argument mining is the retrieval of argumentative content pertaining to a given topic. Most previous work addressed this task by retrieving a relatively small number of relevant documents as the initial source for such content. This line of research yielded moderate success, which is of limited use in a real-world system. Furthermore, for such a system to yield a comprehensive set of relevant arguments, over a wide range of topics, it requires leveraging a large and diverse corpus in an appropriate manner. Here we present a first end-to-end high-precision, corpus-wide argument mining system. This is made possible by combining sentence-level queries over an appropriate indexing of a very large corpus of newspaper articles, with an iterative annotation scheme. This scheme addresses the inherent label bias in the data and pinpoints the regions of the sample space whose manual labeling is required to obtain high-precision among top-ranked candidates. 1 Introduction Starting with the seminal work of Mochales Palau and Moens (2009), argument mining has mainly focused on the following tasks - identifying argumentative text segments within a given document; labeling these text segments according to the type of argument and its stance; and elucidating the discourse relations among the detected arguments. Typically, the considered documents were argumentative in nature, taken from a well defined domain, such as legal documents or student essays. More recently, some attention had been given to the corresponding retrieval task - given a controversial topic, retrieve arguments with a clear stance towards this topic. This is usually done by first retrieving - manually or automatically - documents relevant to the topic, and then using argument mining techniques to identify relevant argumentative segments therein. This documents-based approach was originally explored over Wikipedia (Levy et al. 2014; Rinott et al. 2015), and more recently over the entire Web (Stab et al. 2018). For an argument retrieval system to be of practical use requires: (1) high precision, and (2) wide coverage.


On the Measure of Intelligence

arXiv.org Artificial Intelligence

To make deliberate progress towards more intelligent and more human-like artificial systems, we need to be following an appropriate feedback signal: we need to be able to define and evaluate intelligence in a way that enables comparisons between two systems, as well as comparisons with humans. Over the past hundred years, there has been an abundance of attempts to define and measure intelligence, across both the fields of psychology and AI. We summarize and critically assess these definitions and evaluation approaches, while making apparent the two historical conceptions of intelligence that have implicitly guided them. We note that in practice, the contemporary AI community still gravitates towards benchmarking intelligence by comparing the skill exhibited by AIs and humans at specific tasks such as board games and video games. We argue that solely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experience: unlimited priors or unlimited training data allow experimenters to "buy" arbitrary levels of skills for a system, in a way that masks the system's own generalization power. We then articulate a new formal definition of intelligence based on Algorithmic Information Theory, describing intelligence as skill-acquisition efficiency and highlighting the concepts of scope, generalization difficulty, priors, and experience. Using this definition, we propose a set of guidelines for what a general AI benchmark should look like. Finally, we present a benchmark closely following these guidelines, the Abstraction and Reasoning Corpus (ARC), built upon an explicit set of priors designed to be as close as possible to innate human priors. We argue that ARC can be used to measure a human-like form of general fluid intelligence and that it enables fair general intelligence comparisons between AI systems and humans.


What's New in EDU: Introducing a new Minecraft Hour of Code tutorial with AI and the Discovery STEM Careers Coalition Microsoft EDU

#artificialintelligence

There's a good chance the students you're teaching today will enroll in university courses that haven't yet been created and enter jobs that don't exist. And they'll be called upon to solve some of the world's most pressing environmental, social and economic issues. We know that can feel like a lot on your shoulders, but there is plenty you can do to prepare students for success and we're here to help. Thoughtfully designed and well implemented STEM instruction builds subject-specific knowledge and fosters a growth mindset, collaboration, critical thinking and computational thinking – all vital skills for jobs of the future. We have tips to share about fun ways to participate in Hour of Code, available in Minecraft: Education Edition as a free coding lesson.


4 reasons why HR leaders need to be ready for AI in their workplaces

#artificialintelligence

One of the biggest change agents in organisations today is technology. Technology has changed and continues to change the way organisations are structured, through the likes of virtual teams, remote workplaces and outsourcing of departments to other countries. Ways in which organisations work are also changing, through the likes of online meeting rooms; e-learning, online performance appraisals, video recruiting, virtual customer service platforms, chatbots and e-financial platforms. There are a range of technologies disrupting organisations and HR, but the one that can be considered the biggest is AI. AI in the form of chatbots, learning platforms, machine learning systems, voice recognition systems as well as virtual assistants, are becoming more commonplace in the workplace.


MIT conference focuses on preparing workers for the era of artificial

#artificialintelligence

In opening yesterday's AI and the Work of the Future Congress, MIT Professor Daniela Rus presented diverging views of how artificial intelligence will impact jobs worldwide. By automating certain menial tasks, experts think AI is poised to improve human quality of life, boost profits, and create jobs, said Rus, director of the Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science. Rus then quoted a World Economic Forum study estimating AI could help create 133 million new jobs worldwide over the next five years. Juxtaposing this optimistic view, however, she noted a recent survey that found about two-thirds of Americans believe machines will soon rob humans of their careers. The economists, who predict greater productivity and new jobs?


Prosthetic legs of California high school wrestling captain stolen from gym

FOX News

Fox News Flash top headlines for Nov. 24 are here. Check out what's clicking on FoxNews.com The prosthetic legs of a double amputee and soon-to-be high school wrestling captain were stolen from a gym closet in California last week, putting his dreams of winning a state championship or even wrestling this season in doubt. Brett Winters, a senior at Pacific High School in San Bernardino, California, was born without tibia bones in his legs. As a baby, his mother was told by doctors that Winters could either spend life in a wheelchair or amputate his legs.


To secure a safer future for AI, we need the benefit of a female perspective John Naughton

#artificialintelligence

Everybody knows (or should know) by now that machine learning (which is what most current artificial intelligence actually amounts to) is subject to bias. Last week, the New York Times had the idea of asking three prominent experts in the field to talk about the bias problem, in particular the ways that social bias can be reflected and amplified in dangerous ways by the technology to discriminate against, or otherwise damage, certain social groups. At first sight, the resulting article looked like a run-of-the-mill review of what has become a common topic – except for one thing: the three experts were all women. One, Daphne Koller, is a co-founder of the online education company Coursera; another, Olga Russakovsky, is a Princeton professor who is working to reduce bias in ImageNet, the data set that powered the current machine-learning boom; the third, Timnit Gebru, is a research scientist at Google in the company's ethical AI team. Reading the observations of these three women brought to the surface a thought that's been lurking at the back of my mind for years.