Education
A Survey of Optimization Methods from a Machine Learning Perspective
Sun, Shiliang, Cao, Zehui, Zhu, Han, Zhao, Jing
Machine learning develops rapidly, which has made many theoretical breakthroughs and is widely applied in various fields. Optimization, as an important part of machine learning, has attracted much attention of researchers. With the exponential growth of data amount and the increase of model complexity, optimization methods in machine learning face more and more challenges. A lot of work on solving optimization problems or improving optimization methods in machine learning has been proposed successively. The systematic retrospect and summary of the optimization methods from the perspective of machine learning are of great significance, which can offer guidance for both developments of optimization and machine learning research. In this paper, we first describe the optimization problems in machine learning. Then, we introduce the principles and progresses of commonly used optimization methods. Next, we summarize the applications and developments of optimization methods in some popular machine learning fields. Finally, we explore and give some challenges and open problems for the optimization in machine learning.
A gray-box approach for curriculum learning
Foglino, Francesco, Leonetti, Matteo, Sagratella, Simone, Seccia, Ruggiero
Curriculum learning is often employed in deep reinforcement learning to let the agent progress more quickly towards better behaviors. Numerical methods for curriculum learning in the literature provides only initial heuristic solutions, with little to no guarantee on their quality. We define a new gray-box function that, including a suitable scheduling problem, can be effectively used to reformulate the curriculum learning problem. We propose different efficient numerical methods to address this gray-box reformulation. Preliminary numerical results on a benchmark task in the curriculum learning literature show the viability of the proposed approach.
Advanced Machine Learning with Basic Excel
In this article, I present a few modern techniques that have been used in various business contexts, comparing performance with traditional methods. The advanced techniques in question are math-free, innovative, efficiently process large amounts of unstructured data, and are robust and scalable. Implementations in Python, R, Julia and Perl are provided, but here we focus on an Excel version that does not even require any Excel macros, coding, plug-ins, or anything other than the most basic version of Excel. It is actually easily implemented in standard, basic SQL too, and we invite readers to work on an SQL version. In short, we offer here an Excel template for machine learning and statistical computing, and it is quite powerful for an Excel spreadsheet.
How to make data and AI add up
In a well-worn clichรฉ, data is often referred to as "the new oil". The analogy is limited, but it does have some truth to it as data -- like oil -- is the defining resource for a new industrial age. Likewise, data seems set to be dominated by a small number of massive global players. For organisations hoping to become pioneers in artificial intelligence (AI) and data analytics, scale confers significant competitive advantages. Bigger companies will be better placed to build the bigger data sets that enable more sophisticated analysis to be performed more quickly.
matloff/R-vs.-Python-for-Data-Science
This Web page is aimed at shedding some light on the perennial R-vs.-Python debates in the Data Science community. As a professional computer scientist and statistician, I hope to shed some useful light on the topic. I have potential bias -- I've written 4 R-related books, and currently serve as Editor-in-Chief of the R Journal -- but I hope this analysis will be considered fair and helpful. This is subjective, of course, but having written (and taught) in many different programming languages, I really appreciate Python's greatly reduced use of parentheses and braces: This is of particular interest to me, as an educator. I've taught a number of subjects -- math, stat, CS and even English As a Second Language -- and have given intense thought to the learning process for many, many years.
Machine Learning Institute (Training) Course in Chandigarh
Machine learning future has just begun and you can grab this opportunity to build your future and earn good salary packages in the industry. If you want to become a data scientist or want to lead the team of analysts, enrol in machine learning training in Mohali. We help you to clear your doubts and learn data science techniques, gain expertise in machine learning algorithms. You will learn to handle multi-variety or multi-dimensional data in dynamic environments. Don't get confused to choose your career path as you can build a successful career in machine learning.
It's Time To Demystify Machine Learning
The hype machine is cranked up to an 11 on the topic of machine learning (sometimes called artificial intelligence, though I don't call it that because AI is not really intelligence and there's nothing artificial about it). Machine learning will either empower the world or take it over, depending on what you read. But before you get swept away by the gust of hot air coming from the technology industry, it's important to pause in order to put things into perspective. Maybe just explaining it in reasonable terms will help. Shortly after the first caveman figured out how to make fire, the second caveman wanted to learn how to make fire, too.
Robust Federated Learning in a Heterogeneous Environment
Ghosh, Avishek, Hong, Justin, Yin, Dong, Ramchandran, Kannan
We study a recently proposed large-scale distributed learning paradigm, namely Federated Learning, where the worker machines are end users' own devices. Statistical and computational challenges arise in Federated Learning particularly in the presence of heterogeneous data distribution (i.e., data points on different devices belong to different distributions signifying different clusters) and Byzantine machines (i.e., machines that may behave abnormally, or even exhibit arbitrary and potentially adversarial behavior). To address the aforementioned challenges, first we propose a general statistical model for this problem which takes both the cluster structure of the users and the Byzantine machines into account. Then, leveraging the statistical model, we solve the robust heterogeneous Federated Learning problem \emph{optimally}; in particular our algorithm matches the lower bound on the estimation error in dimension and the number of data points. Furthermore, as a by-product, we prove statistical guarantees for an outlier-robust clustering algorithm, which can be considered as the Lloyd algorithm with robust estimation. Finally, we show via synthetic as well as real data experiments that the estimation error obtained by our proposed algorithm is significantly better than the non-Byzantine-robust algorithms; in particular, we gain at least by 53\% and 33\% for synthetic and real data experiments, respectively, in typical settings.
While We Wait For Artificial Superintelligence, Let's Make The Most Of Augmented Intelligence
The increasing buzz of artificial intelligence in news and science fiction generally creates an illusion that machines might imitate and surpass human intelligence. However, that is only a type of artificial intelligence called artificial superintelligence. Artificial superintelligence is something that can be seen in sci-fi movies like Interstellar, where TARS assists the astronauts in their space mission along with having human-like conversations. However, there are also other sci-fi fantasies like the one in'Avengers: Age of Ultron,' where the artificial superintelligence-based antagonist decides to wipe out humanity itself. Such scenarios are concerning people and even tech leaders like Bill Gates and Elon Musk who have warned against the expansion of AI.