Goto

Collaborating Authors

 Education


Robust Federated Learning in a Heterogeneous Environment

arXiv.org Machine Learning

We study a recently proposed large-scale distributed learning paradigm, namely Federated Learning, where the worker machines are end users' own devices. Statistical and computational challenges arise in Federated Learning particularly in the presence of heterogeneous data distribution (i.e., data points on different devices belong to different distributions signifying different clusters) and Byzantine machines (i.e., machines that may behave abnormally, or even exhibit arbitrary and potentially adversarial behavior). To address the aforementioned challenges, first we propose a general statistical model for this problem which takes both the cluster structure of the users and the Byzantine machines into account. Then, leveraging the statistical model, we solve the robust heterogeneous Federated Learning problem \emph{optimally}; in particular our algorithm matches the lower bound on the estimation error in dimension and the number of data points. Furthermore, as a by-product, we prove statistical guarantees for an outlier-robust clustering algorithm, which can be considered as the Lloyd algorithm with robust estimation. Finally, we show via synthetic as well as real data experiments that the estimation error obtained by our proposed algorithm is significantly better than the non-Byzantine-robust algorithms; in particular, we gain at least by 53\% and 33\% for synthetic and real data experiments, respectively, in typical settings.


While We Wait For Artificial Superintelligence, Let's Make The Most Of Augmented Intelligence

#artificialintelligence

The increasing buzz of artificial intelligence in news and science fiction generally creates an illusion that machines might imitate and surpass human intelligence. However, that is only a type of artificial intelligence called artificial superintelligence. Artificial superintelligence is something that can be seen in sci-fi movies like Interstellar, where TARS assists the astronauts in their space mission along with having human-like conversations. However, there are also other sci-fi fantasies like the one in'Avengers: Age of Ultron,' where the artificial superintelligence-based antagonist decides to wipe out humanity itself. Such scenarios are concerning people and even tech leaders like Bill Gates and Elon Musk who have warned against the expansion of AI.


Transmedia Storytelling Initiative launches with $1.1 million gift

#artificialintelligence

Driven by the rise of transformative digital technologies and the proliferation of data, human storytelling is rapidly evolving in ways that challenge and expand our very understanding of narrative. Transmedia -- where stories and data operate across multiple platforms and social transformations -- and its wide range of theoretical, philosophical, and creative perspectives, needs shared critique around making and understanding. MIT's School of Architecture and Planning (SA P), working closely with faculty in the MIT School of Humanities, Arts, and Social Sciences (SHASS) and others across the Institute, has launched the Transmedia Storytelling Initiative under the direction of Professor Caroline Jones, an art historian, critic, and curator in the History, Theory, Criticism section of SA P's Department of Architecture. The initiative will build on MIT's bold tradition of art education, research, production, and innovation in media-based storytelling, from film through augmented reality. Supported by a foundational gift from David and Nina Fialkow, this initiative will create an influential hub for pedagogy and research in time-based media.


ยฃ18.5 million to boost diversity in AI tech roles and innovation in online training for adults

#artificialintelligence

The technology sector is set to benefit from a ยฃ18.5 million cash injection to drive up skills in AI and data science and support more adults to upskill and retrain to progress in their careers or find new employment. Up to 2,500 people will have the opportunity to retrain and become experts in data science and artificial intelligence (AI), thanks to a ยฃ13.5 million investment to fund new degree and Masters conversion courses and scholarships at UK academic institutions over the next three years. The ground-breaking Adult Learning Technology Innovation Fund, which will be launched in partnership with innovation foundation Nesta, will provide funding and expertise to incentivise tech firms to harness new technologies to develop bespoke, flexible, inclusive, and engaging online training opportunities to support more people into skilled employment. Companies across the tech sector already employ more than 2.1 million people, contribute ยฃ184 billion to the economy every year and inward investment to the UK AI sector stood at ยฃ1 billion for 2018, which is more than Germany, France, Netherlands, Sweden and Switzerland combined. To further strengthen the sector, Government is investing in data-driven technologies, such as artificial intelligence, through the modern Industrial Strategy, so tech businesses and people with the drive and talent can succeed.


Build and scale your business' AI capabilities

#artificialintelligence

Artificial Intelligence and Machine Learning are two of the most popular and powerful technologies in the world at preset. In fact, it is believed that the market of AI and ML is forecasted to grow in huge numbers. In fact, even the voice searches are forecasted to grow at approximately 270% across India. In fact, AI is expected to touch even the most unexpected areas across the world. For example, Agricultural industry, astronomy and the field of defense are a few of the areas where we didn't expect AI to make such huge impact.


Memozing - E-learning Network

#artificialintelligence

Learn faster, learn better, learn easier, and learn with more fun. Want to learn a new language, must learn something for school or university, or want to train yourself in a profession? Then you are in the right place. Our patented learning project helps you structure your knowledge area and promotes every knowledge area to be guaranteed in the long-term memory. Learn exactly what you want.


SQIL: Imitation Learning via Regularized Behavioral Cloning

arXiv.org Machine Learning

Learning to imitate expert behavior given action demonstrations containing high-dimensional, continuous observations and unknown dynamics is a difficult problem in robotic control. Simple approaches based on behavioral cloning (BC) suffer from state distribution shift, while more complex methods that generalize to out-of-distribution states can be difficult to use, since they typically involve adversarial optimization. We propose an alternative that combines the simplicity of BC with the robustness of adversarial imitation learning. The key insight is that under the maximum entropy model of expert behavior, BC corresponds to fitting a soft Q function that maximizes the likelihood of observed actions. This perspective suggests a way to regularize BC so that it generalizes to out-of-distribution states: combine the standard maximum-likelihood objective with a penalty on the soft Bellman error of the soft Q function. We show that this penalty term gives the agent an incentive to take actions that lead it back to demonstrated states when it encounters new states. Experiments show that our method outperforms BC and GAIL on a variety of image-based and low-dimensional environments in Box2D, Atari, and MuJoCo.


Support vector machines on the D-Wave quantum annealer

arXiv.org Machine Learning

Kernel-based support vector machines (SVMs) are supervised machine learning algorithms for classification and regression problems. We present a method to train SVMs on a D-Wave 2000Q quantum annealer and study its performance in comparison to SVMs trained on conventional computers. The method is applied to both synthetic data and real data obtained from biology experiments. We find that the quantum annealer produces an ensemble of different solutions that often generalizes better to unseen data than the single global minimum of an SVM trained on a conventional computer, especially in cases where only limited training data is available. For cases with more training data than currently fits on the quantum annealer, we show that a combination of classifiers for subsets of the data almost always produces stronger joint classifiers than the conventional SVM for the same parameters.


Amortized Bethe Free Energy Minimization for Learning MRFs

arXiv.org Machine Learning

We propose to learn deep undirected graphical models (i.e., MRFs), with a non-ELBO objective for which we can calculate exact gradients. In particular, we optimize a saddle-point objective deriving from the Bethe free energy approximation to the partition function. Unlike much recent work in approximate inference, the derived objective requires no sampling, and can be efficiently computed even for very expressive MRFs. We furthermore amortize this optimization with trained inference networks. Experimentally, we find that the proposed approach compares favorably with loopy belief propagation, but is faster, and it allows for attaining better held out log likelihood than other recent approximate inference schemes.


Identify treatment effect patterns for personalised decisions

arXiv.org Machine Learning

In personalised decision making, evidence is required to determine suitable actions for individuals. Such evidence can be obtained by identifying treatment effect heterogeneity in different subgroups of the population. In this paper, we design a new type of pattern, treatment effect pattern to represent and discover treatment effect heterogeneity from data for determining whether a treatment will work for an individual or not. Our purpose is to use the computational power to find the most specific and relevant conditions for individuals with respect to a treatment or an action to assist with personalised decision making. Most existing work on identifying treatment effect heterogeneity takes a top-down or partitioning based approach to search for subgroups with heterogeneous treatment effects. We propose a bottom-up generalisation algorithm to obtain the most specific patterns that fit individual circumstances the best for personalised decision making. For the generalisation, we follow a consistency driven strategy to maintain inner-group homogeneity and inter-group heterogeneity of treatment effects. We also employ graphical causal modelling technique to identify adjustment variables for reliable treatment effect pattern discovery. Our method can find the treatment effect patterns reliably as validated by the experiments. The method is faster than the two existing machine learning methods for heterogeneous treatment effect identification and it produces subgroups with higher inner-group treatment effect homogeneity.