The minimum description length (MDL) principle in supervised learning is studied. One of the most important theories for the MDL principle is Barron and Cover's theory (BC theory), which gives a mathematical justification of the MDL principle. The original BC theory, however, can be applied to supervised learning only approximately and limitedly. Though Barron et al. recently succeeded in removing a similar approximation in case of unsupervised learning, their idea cannot be essentially applied to supervised learning in general. To overcome this issue, an extension of BC theory to supervised learning is proposed. The derived risk bound has several advantages inherited from the original BC theory. First, the risk bound holds for finite sample size. Second, it requires remarkably few assumptions. Third, the risk bound has a form of redundancy of the two-stage code for the MDL procedure. Hence, the proposed extension gives a mathematical justification of the MDL principle to supervised learning like the original BC theory. As an important example of application, new risk and (probabilistic) regret bounds of lasso with random design are derived. The derived risk bound holds for any finite sample size $n$ and feature number $p$ even if $n\ll p$ without boundedness of features in contrast to the past work. Behavior of the regret bound is investigated by numerical simulations. We believe that this is the first extension of BC theory to general supervised learning with random design without approximation.

This post was first published on my Linkedin page and posted here as a contributed post. In the last few months, I have had several people contact me about their enthusiasm for venturing into the world of data science and using Machine Learning (ML) techniques to probe statistical regularities and build impeccable data-driven products. However, I've observed that some actually lack the necessary mathematical intuition and framework to get useful results. This is the main reason I decided to write this blog post. Recently, there has been an upsurge in the availability of many easy-to-use machine and deep learning packages such as scikit-learn, Weka, Tensorflow etc. Machine Learning theory is a field that intersects statistical, probabilistic, computer science and algorithmic aspects arising from learning iteratively from data and finding hidden insights which can be used to build intelligent applications. Despite the immense possibilities of Machine and Deep Learning, a thorough mathematical understanding of many of these techniques is necessary for a good grasp of the inner workings of the algorithms and getting good results.

Machine learning algorithms is one of the hot topic in today' s world. We are started our journey from Mainframe computers to PC and now in CLOUD computing. The definition says: Machine learning is a sub field of computer science that evolved from the study of pattern recognition and computational learning theory in artificial intelligence. In 1959, Arthur Samuel defined machine learning as a "Field of study that gives computers the ability to learn without being explicitly programmed" Supervised learning: This algorithm consist of a target / outcome variable (or dependent variable) which is to be predicted from a given set of predictors (independent variables). Un-supervised learning: In this algorithm, we do not have any target or outcome variable to predict / estimate.

Consider a toddler navigating her day, bombarded by a kaleidoscope of experiences. How does her mind discover what's normal happenstance and begin building a model of the world? How does she recognize unusual events and incorporate them into her worldview? How does she understand new concepts, often from just a single example?

In this presentation, Mate plans to talk about the base of all NP-complete problems, the satisfiability problem, how these problems are being solved with state-of-the-art SAT solvers, and how that is relevant to our everyday lives. SAT solvers have enjoyed a boom thanks to massive improvements through the use of lazy data structures, tricky graph algorithms, proof verification, and heuristics. This allowed SAT solvers to thrive in many different tasks such as hardware verification,software fuzzing, cryptography, and math proofs. In this talk we will examine some of the core data structures and algorithms used by SAT solvers, as well as show some example use-cases for them in different fields.

Taking a Machine Learning for Data Science and Analytics course online and I am having trouble understanding those two topics and being able to differentiate between them. I understand that some of this may be considered "Computer Science Theory." Hopefully someone can still answer this question.

At ICML last year and the year before the amount of capacity that needed to fit everyone on any single day was about 1500. My advice was to expect 2000 and have capacity for 2500 because "New York" and "Machine Learning". I was not involved in the venue negotiations, but my understanding is that they were difficult, with liabilities over 1M for IMLS the nonprofit which oversees ICML year to year. The result was a conference plan with a maximum capacity of 1800 for the main conference, a bit less for workshops, and perhaps 1000 for tutorials. Then the NIPS registration numbers came in: 3900 last winter.

The best we can hope for when it comes to most decisions is to be probably approximately correct–a high probability of being about right. In finance, analysts compare proposed capital costs with discounted anticipated future cash flows to calculate a net present value–a bunch of assumptions with the hope of being probably approximately correct. Insurance is a hedge against a big loss; it's based on the probability of bad stuff happening–the insurance company makes a little money if their calculations are probably approximately correct. A doctor takes a few data points and makes a diagnosis hoping she is probably approximately correct. School facilities planners estimate future enrollment trends and then school boards estimate the likelihood of a community support for a construction bond, both hope to be probably approximately correct.

We give a complete characterization of the complexity of best-arm identification in one-parameter bandit problems. We prove a new, tight lower bound on the sample complexity. We propose the `Track-and-Stop' strategy, which we prove to be asymptotically optimal. It consists in a new sampling rule (which tracks the optimal proportions of arm draws highlighted by the lower bound) and in a stopping rule named after Chernoff, for which we give a new analysis.

In episode ten of season two, we talk about Computational Learning Theory and Probably Approximately Correct Learning originated by Professor Leslie Valiant of SEAS at Harvard, we take a listener question about generative systems, plus we talk with Aviv Regev, Chair of the Faculty and Director of the Klarman Cell Observatory and the Cell Circuits Program at the Broad Institute.