Goto

Collaborating Authors

 Education


A general framework for decentralized optimization with first-order methods

arXiv.org Machine Learning

Decentralized optimization to minimize a finite sum of functions over a network of nodes has been a significant focus within control and signal processing research due to its natural relevance to optimal control and signal estimation problems. More recently, the emergence of sophisticated computing and large-scale data science needs have led to a resurgence of activity in this area. In this article, we discuss decentralized first-order gradient methods, which have found tremendous success in control, signal processing, and machine learning problems, where such methods, due to their simplicity, serve as the first method of choice for many complex inference and training tasks. In particular, we provide a general framework of decentralized first-order methods that is applicable to undirected and directed communication networks alike, and show that much of the existing work on optimization and consensus can be related explicitly to this framework. We further extend the discussion to decentralized stochastic first-order methods that rely on stochastic gradients at each node and describe how local variance reduction schemes, previously shown to have promise in the centralized settings, are able to improve the performance of decentralized methods when combined with what is known as gradient tracking. We motivate and demonstrate the effectiveness of the corresponding methods in the context of machine learning and signal processing problems that arise in decentralized environments.


Learning from Very Few Samples: A Survey

arXiv.org Machine Learning

Few sample learning (FSL) is significant and challenging in the field of machine learning. The capability of learning and generalizing from very few samples successfully is a noticeable demarcation separating artificial intelligence and human intelligence since humans can readily establish their cognition to novelty from just a single or a handful of examples whereas machine learning algorithms typically entail hundreds or thousands of supervised samples to guarantee generalization ability. Despite the long history dated back to the early 2000s and the widespread attention in recent years with booming deep learning technologies, little surveys or reviews for FSL are available until now. In this context, we extensively review 300+ papers of FSL spanning from the 2000s to 2019 and provide a timely and comprehensive survey for FSL. In this survey, we review the evolution history as well as the current progress on FSL, categorize FSL approaches into the generative model based and discriminative model based kinds in principle, and emphasize particularly on the meta learning based FSL approaches. We also summarize several recently emerging extensional topics of FSL and review the latest advances on these topics. Furthermore, we highlight the important FSL applications covering many research hotspots in computer vision, natural language processing, audio and speech, reinforcement learning and robotic, data analysis, etc. Finally, we conclude the survey with a discussion on promising trends in the hope of providing guidance and insights to follow-up researches.


MIT hosts seven distinguished MLK Professors and Scholars for 2020-21

#artificialintelligence

In light of the Covid-19 pandemic, MIT has been charged with reimagining its campus, classes, and programs, including the Dr. Martin Luther King, Jr. (MLK) Visiting Professors and Scholars Program (VPSP). Founded in 1990, MLK VPSP honors the life and legacy of Martin Luther King, Jr. by increasing the presence of and recognizing the contributions of scholars from underrepresented groups at MIT. MLK Visiting Professors and Scholars enhance their scholarship through intellectual engagement with the MIT community and enrich the cultural, academic, and professional experience of students. But what does a virtual year mean for a visiting scholar? Even with the challenge of remote learning and limited in-person contact, MLK VPSP faculty hosts have articulated innovative ways to engage with the MIT community. Moya Bailey, for instance, will be a content contributor for the Program in Women's and Gender Studies' website and social media accounts.


AI and Mentoring: The Perfect Match

#artificialintelligence

If ever there was a time for mentorship, it's now. Even back in the days of'business as usual', employees valued having someone to lean on – an experienced colleague who could offer support and advice on tricky situations, business decisions and career progression. It was easier then; all you had to do was knock on their office door or drop them an email requesting a catch up over coffee. But since COVID-19 rocked up, two things have happened: traditional avenues for accessing mentors have all but vanished, and the trend of remote working has stripped employees of vital support networks and structure, making mentoring more important than ever. Some people roll their eyes at the very mention of mentorships; they write them off as a box-ticking exercise for interns or a pastime for chief executives with too much time on their hands (as if such a thing existed). For those people, let me set the record straight.


Charlene Chambliss: From Psychology to Natural Language Processing and Applied Research

#artificialintelligence

Interest in data science has been exponentially increasing over the past decade, and more and more people are working towards making a career switch into the field. In 2020, articles and YouTube videos about transitioning into a career in data science abound. Yet, for a lot of people, many key questions about this switch still remain: How do you break into data science from a social science background? And what are some of the most important skills in fields like psychology that can be applied to data science? Charlene Chambliss has an inspiring and non-traditional career path.


Satellite Data Fill the Void of Dwindling Crop Tours

#artificialintelligence

The pandemic is helping to usher in a new era of food-production forecasts that rely more on satellite data and artificial intelligence and less on information gathered by people. The crop world, including major trading houses and statisticians at the U.S. Department of Agriculture, has long depended on scouts trudging through fields to count corn kernels and soybean pods. But travel restrictions and new virus safety measures have cut participation in field tours at a time of increasing scrutiny over food security. "Covid-19 is disrupting agricultural supply chains in developing countries, and observers on the ground can no longer report on crop conditions," said Lillian Kay Petersen, a student at Harvard University. She won the top prize of this year's Regeneron Science Talent Search, a 79-year-old competition for high school students held by the Society for Science and the Public, for her model that uses daily satellite images to predict crop yields in Africa.


Top 10 B Tech Colleges for Data Science

#artificialintelligence

Data Analytics, Big data, Deep Learning covering some aspects of Hadoop - EcoSystem and AI, so far was being introduced in small dosage and capsules in many campuses - IIT / NIT and other engineering colleges. Few days to week long workshops or crash course short term training programs and FDP programs for faculty members in association with industry has been going around which gave the engineering students or faculties a basic idea and opportunity to upgrade their knowledge in the areas of recent emerging technologies. Artificial Intelligence & Machine Learning; Internet of Things (IOT) Using Amazon AWS; Blockchain Technology; Data Science Using Python are among the common and popularly held workshops on campuses. But with the influence and usage of Big Data, AI (Artificial Intelligence) and Data Analytics across business and industry - big or small, many colleges have started introducing these courses as full time programs. Indian Institute of Technology (IIT) Mandi has become the first IIT to launch a full-fledged Bachelor's Programme in Data Science and Engineering (B Tech in Data Science).


Applications of Deep Neural Networks

arXiv.org Artificial Intelligence

Deep learning is a group of exciting new technologies for neural networks. Through a combination of advanced training techniques and neural network architectural components, it is now possible to create neural networks that can handle tabular data, images, text, and audio as both input and output. Deep learning allows a neural network to learn hierarchies of information in a way that is like the function of the human brain. This course will introduce the student to classic neural network structures, Convolution Neural Networks (CNN), Long Short-Term Memory (LSTM), Gated Recurrent Neural Networks (GRU), General Adversarial Networks (GAN), and reinforcement learning. Application of these architectures to computer vision, time series, security, natural language processing (NLP), and data generation will be covered. High-Performance Computing (HPC) aspects will demonstrate how deep learning can be leveraged both on graphical processing units (GPUs), as well as grids. Focus is primarily upon the application of deep learning to problems, with some introduction to mathematical foundations. Readers will use the Python programming language to implement deep learning using Google TensorFlow and Keras. It is not necessary to know Python prior to this book; however, familiarity with at least one programming language is assumed.


Large-scale empirical validation of Bayesian Network structure learning algorithms with noisy data

arXiv.org Artificial Intelligence

Numerous Bayesian Network (BN) structure learning algorithms have been proposed in the literature over the past few decades. Each publication makes an empirical or theoretical case for the algorithm proposed in that publication and results across studies are often inconsistent in their claims about which algorithm is 'best'. This is partly because there is no agreed evaluation approach to determine their effectiveness. Moreover, each algorithm is based on a set of assumptions, such as complete data and causal sufficiency, and tend to be evaluated with data that conforms to these assumptions, however unrealistic these assumptions may be in the real world. As a result, it is widely accepted that synthetic performance overestimates real performance, although to what degree this may happen remains unknown. This paper investigates the performance of 15 structure learning algorithms. We propose a methodology that applies the algorithms to data that incorporates synthetic noise, in an effort to better understand the performance of structure learning algorithms when applied to real data. Each algorithm is tested over multiple case studies, sample sizes, types of noise, and assessed with multiple evaluation criteria. This work involved approximately 10,000 graphs with a total structure learning runtime of seven months. It provides the first large-scale empirical validation of BN structure learning algorithms under different assumptions of data noise. The results suggest that traditional synthetic performance may overestimate real-world performance by anywhere between 10% and more than 50%. They also show that while score-based learning is generally superior to constraint-based learning, a higher fitting score does not necessarily imply a more accurate causal graph. To facilitate comparisons with future studies, we have made all data, raw results, graphs and BN models freely available online.


Teaching Tech to Talk: K-12 Conversational Artificial Intelligence Literacy Curriculum and Development Tools

arXiv.org Artificial Intelligence

With children talking to smart-speakers, smart-phones and even smart-microwaves daily, it is increasingly important to educate students on how these agents work-from underlying mechanisms to societal implications. Researchers are developing tools and curriculum to teach K-12 students broadly about artificial intelligence (AI); however, few studies have evaluated these tools with respect to AI-specific learning outcomes, and even fewer have addressed student learning about AI-based conversational agents. We evaluate our Conversational Agent Interface for MIT App Inventor and workshop curriculum with respect to eight AI competencies from the literature. Furthermore, we analyze teacher (n=9) and student (n=47) feedback from workshops with the interface and recommend that future work leverages design considerations from the literature to optimize engagement, collaborates with teachers, and addresses a range of student abilities through pacing and opportunities for extension. We found students struggled most with the concepts of AI ethics and learning, and recommend emphasizing these topics when teaching. The appendix, including a demo video, can be found here: https://gist.github.com/jessvb/1cd959e32415a6ad4389761c49b54bbf