Some machine learning applications involve training data that is sensitive, such as the medical histories of patients in a clinical trial. A model may inadvertently and implicitly store some of its training data; careful analysis of the model may therefore reveal sensitive information. To address this problem, we demonstrate a generally applicable approach to providing strong privacy guarantees for training data: Private Aggregation of Teacher Ensembles (PATE). The approach combines, in a black-box fashion, multiple models trained with disjoint datasets, such as records from different subsets of users. Because they rely directly on sensitive data, these models are not published, but instead used as "teachers" for a "student" model. The student learns to predict an output chosen by noisy voting among all of the teachers, and cannot directly access an individual teacher or the underlying data or parameters. The student's privacy properties can be understood both intuitively (since no single teacher and thus no single dataset dictates the student's training) and formally, in terms of differential privacy. These properties hold even if an adversary can not only query the student but also inspect its internal workings. Compared with previous work, the approach imposes only weak assumptions on how teachers are trained: it applies to any model, including non-convex models like DNNs. We achieve state-of-the-art privacy/utility trade-offs on MNIST and SVHN thanks to an improved privacy analysis and semi-supervised learning.

Matrix factorization is a popular approach to solving matrix estimation problems based on partial observations. Existing matrix factorization is based on least squares and aims to yield a low-rank matrix to interpret the conditional sample means given the observations. However, in many real applications with skewed and extreme data, least squares cannot explain their central tendency or tail distributions, yielding undesired estimates. In this paper, we propose \emph{expectile matrix factorization} by introducing asymmetric least squares, a key concept in expectile regression analysis, into the matrix factorization framework. We propose an efficient algorithm to solve the new problem based on alternating minimization and quadratic programming. We prove that our algorithm converges to a global optimum and exactly recovers the true underlying low-rank matrices when noise is zero. For synthetic data with skewed noise and a real-world dataset containing web service response times, the proposed scheme achieves lower recovery errors than the existing matrix factorization method based on least squares in a wide range of settings.

We introduce Deep Variational Bayes Filters (DVBF), a new method for unsupervised learning and identification of latent Markovian state space models. Leveraging recent advances in Stochastic Gradient Variational Bayes, DVBF can overcome intractable inference distributions via variational inference. Thus, it can handle highly nonlinear input data with temporal and spatial dependencies such as image sequences without domain knowledge. Our experiments show that enabling backpropagation through transitions enforces state space assumptions and significantly improves information content of the latent embedding. This also enables realistic long-term prediction.

This paper provides the first empirical demonstration that deep convolutional models really need to be both deep and convolutional, even when trained with methods such as distillation that allow small or shallow models of high accuracy to be trained. Although previous research showed that shallow feed-forward nets sometimes can learn the complex functions previously learned by deep nets while using the same number of parameters as the deep models they mimic, in this paper we demonstrate that the same methods cannot be used to train accurate models on CIFAR-10 unless the student models contain multiple layers of convolution. Although the student models do not have to be as deep as the teacher model they mimic, the students need multiple convolutional layers to learn functions of comparable accuracy as the deep convolutional teacher.

In this paper, we study the information-theoretic limits of learning the structure of Bayesian networks (BNs), on discrete as well as continuous random variables, from a finite number of samples. We show that the minimum number of samples required by any procedure to recover the correct structure grows as $\Omega(m)$ and $\Omega(k \log m + (k^2/m))$ for non-sparse and sparse BNs respectively, where $m$ is the number of variables and $k$ is the maximum number of parents per node. We provide a simple recipe, based on an extension of the Fano's inequality, to obtain information-theoretic limits of structure recovery for any exponential family BN. We instantiate our result for specific conditional distributions in the exponential family to characterize the fundamental limits of learning various commonly used BNs, such as conditional probability table based networks, gaussian BNs, noisy-OR networks, and logistic regression networks. En route to obtaining our main results, we obtain tight bounds on the number of sparse and non-sparse essential-DAGs. Finally, as a byproduct, we recover the information-theoretic limits of sparse variable selection for logistic regression.

March 2, 2017 --Computer engineers have created some amazingly small devices, capable of storing entire libraries of music and movies in the palm of your hand. But geneticists say Mother Nature can do even better. DNA, where all of biology's information is stored, is incredibly dense. The whole genome of an organism fits into a cell that is invisible to the naked eye. That's why computer scientists are turning to microbiology to design the next best way to store humanity's ever-increasing collection of digital data.

SYS-CON Events announced today that TechTarget has been named "Media Sponsor" of SYS-CON's 20th International Cloud Expo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. TechTarget storage websites are the best online information resource for news, tips and expert advice for the storage, backup and disaster recovery markets. By creating abundant, high-quality editorial content across more than 140 highly targeted technology-specific websites, TechTarget attracts and nurtures communities of technology buyers researching their companies' information technology needs. By understanding these buyers' content consumption behaviors, TechTarget creates the purchase intent insights that fuel efficient and effective marketing and sales activities for clients around the world. All major researchers estimate there will be tens of billions devices - computers, smartphones, tablets, and sensors - connected to the Internet by 2020.

Do students learn as much when they read digitally as they do in print? For both parents and teachers, knowing whether computer-based media are improving or compromising education is a question of concern. With the surge in popularity of e-books, online learning and open educational resources, investigators have been trying to determine whether students do as well when reading an assigned text on a digital screen as on paper. The answer to the question, however, needs far more than a yes-no response. In my research, I have compared the ways in which we read in print and onscreen.

Scientists have begun actively preparing for the end of the world. A group of experts met in the Arizona desert to discuss potential nightmare scenarios that could take place in the future and how humanity might handle them, Bloomberg reported Thursday. Funded by Tesla co-founder Elon Musk and Skype co-founder Jaan Tallinn, the meeting was comprised of 40 scientists, policy analysts and cyber security experts. The group was tasked with coming up with increasingly likely "doomsday scenarios." "There is huge potential for AI (artificial intelligence) to transform so many aspects of our society in so many ways. At the same time, there are rough edges and potential downsides, like any technology," Eric Horvitz, an artificial intelligence scientist and the co-organizer of the meeting, told Bloomberg.

How do you plan to kick off your summer this year? How do you plan to kick off your summer this year? If you're like me, you will attend the Starmus Festival Of Science And Arts in gorgeous Trondheim, Norway, between 18 – 23 June. The Starmus -- "stars" and "music" -- Science and Arts Festival was the brain child of astrophysicist Brian May, lead guitarist for the rock band Queen, and his dissertation advisor, Garik Israelian, an astrophysicist who led the team that discovered the first observational evidence that supernova explosions are responsible for the formation of stellar mass black holes. This festival is designed to bring together the sciences -- astronomy, physics, and the biological sciences -- with art, film, music and other performing arts, specifically to communicate science to the public.