Education
The 3 Biggest Mistakes on Learning Data Science
I've discussed parts of what I'm going to mention here in other articles, but now I want to give a few directions on what's not data science and how not to learn it. So let's start with the basics. Data science not just knowing some programming languages, math, statistics and have "domain knowledge". We've created a new field, or something like that. There's a lot of things to say and study in this field.
The future of women at work: Transitions in the age of automation
The age of automation, and on the near horizon, artificial intelligence (AI) technologies offer new job opportunities and avenues for economic advancement, but women face new challenges overlaid on long-established ones. Between 40 million and 160 million women globally may need to transition between occupations by 2030, often into higher-skilled roles. To weather this disruption, women (and men) need to be skilled, mobile, and tech-savvy, but women face pervasive barriers on each, and will need targeted support to move forward in the world of work. A new McKinsey Global Institute (MGI) report, The future of women at work: Transitions in the age of automation (PDF–2MB), finds that if women make these transitions, they could be on the path to more productive, better-paid work. If they cannot, they could face a growing wage gap or be left further behind when progress toward gender parity in work is already slow. This new research explores potential patterns in "jobs lost" (jobs displaced by automation), "jobs gained" (job creation driven by economic growth, investment, demographic changes, and technological innovation), and "jobs changed" (jobs whose activities and skill requirements change from partial automation) for women by exploring several scenarios of how automation adoption and job creation trends could play out by 2030 for men and women given current gender patterns in the global workforce. These scenarios are not meant to predict the future; rather, they serve as a tool to understand a range of possible outcomes and identify interventions needed.
Flexibly Fair Representation Learning by Disentanglement
Creager, Elliot, Madras, David, Jacobsen, Jörn-Henrik, Weis, Marissa A., Swersky, Kevin, Pitassi, Toniann, Zemel, Richard
We consider the problem of learning representations that achieve group and subgroup fairness with respect to multiple sensitive attributes. Taking inspiration from the disentangled representation learning literature, we propose an algorithm for learning compact representations of datasets that are useful for reconstruction and prediction, but are also \emph{flexibly fair}, meaning they can be easily modified at test time to achieve subgroup demographic parity with respect to multiple sensitive attributes and their conjunctions. We show empirically that the resulting encoder---which does not require the sensitive attributes for inference---enables the adaptation of a single representation to a variety of fair classification tasks with new target labels and subgroup definitions.
Vertex Classification on Weighted Networks
Helm, Hayden, Vogelstein, Joshua, Priebe, Carey
This paper proposes a discrimination technique for vertices in a weighted network. We assume that the edge weights and adjacencies in the network are conditionally independent and that both sources of information encode class membership information. In particular, we introduce a edge weight distribution matrix to the standard K-Block Stochastic Block Model to model weighted networks. This allows us to develop simple yet powerful extensions of classification techniques using the spectral embedding of the unweighted adjacency matrix. We consider two assumptions on the edge weight distributions and propose classification procedures in both settings. We show the effectiveness of the proposed classifiers by comparing them to quadratic discriminant analysis following the spectral embedding of a transformed weighted network. Moreover, we discuss and show how the methods perform when the edge weights do not encode class membership information.
Learning Clustered Representation for Complex Free Energy Landscapes
Zhang, Jun, Lei, Yao-Kun, Che, Xing, Zhang, Zhen, Yang, Yi Isaac, Gao, Yi Qin
In this paper we first analyzed the inductive bias underlying the data scattered across complex free energy landscapes (FEL), and exploited it to train deep neural networks which yield reduced and clustered representation for the FEL. Our parametric method, called Information Distilling of Metastability (IDM), is end-to-end differentiable thus scalable to ultra-large dataset. IDM is also a clustering algorithm and is able to cluster the samples in the meantime of reducing the dimensions. Besides, as an unsupervised learning method, IDM differs from many existing dimensionality reduction and clustering methods in that it neither requires a cherry-picked distance metric nor the ground-true number of clusters, and that it can be used to unroll and zoom-in the hierarchical FEL with respect to different timescales. Through multiple experiments, we show that IDM can achieve physically meaningful representations which partition the FEL into well-defined metastable states hence are amenable for downstream tasks such as mechanism analysis and kinetic modeling.
Deep Semi-Supervised Anomaly Detection
Ruff, Lukas, Vandermeulen, Robert A., Görnitz, Nico, Binder, Alexander, Müller, Emmanuel, Müller, Klaus-Robert, Kloft, Marius
Deep approaches to anomaly detection have recently shown promising results over shallow approaches on high-dimensional data. Typically anomaly detection is treated as an unsupervised learning problem. In practice however, one may have---in addition to a large set of unlabeled samples---access to a small pool of labeled samples, e.g. a subset verified by some domain expert as being normal or anomalous. Semi-supervised approaches to anomaly detection make use of such labeled data to improve detection performance. Few deep semi-supervised approaches to anomaly detection have been proposed so far and those that exist are domain-specific. In this work, we present Deep SAD, an end-to-end methodology for deep semi-supervised anomaly detection. Using an information-theoretic perspective on anomaly detection, we derive a loss motivated by the idea that the entropy for the latent distribution of normal data should be lower than the entropy of the anomalous distribution. We demonstrate in extensive experiments on MNIST, Fashion-MNIST, and CIFAR-10 along with other anomaly detection benchmark datasets that our approach is on par or outperforms shallow, hybrid, and deep competitors, even when provided with only few labeled training data.
Gradual Machine Learning for Aspect-level Sentiment Analysis
Wang, Yanyan, Chen, Qun, Shen, Jiquan, Hou, Boyi, Ahmed, Murtadha, Li, Zhanhuai
The state-of-the-art solutions for Aspect-Level Sentiment Analysis (ALSA) are built on a variety of deep neural networks (DNN), whose efficacy depends on large amounts of accurately labeled training data. Unfortunately, high-quality labeled training data usually require expensive manual work, and are thus not readily available in many real scenarios. In this paper, we aim to enable effective machine labeling for ALSA without the requirement for manual labeling effort. Towards this aim, we present a novel solution based on the recently proposed paradigm of gradual machine learning. It begins with some easy instances in an ALSA task, which can be automatically labeled by the machine with high accuracy, and then gradually labels the more challenging instances by iterative factor graph inference. In the process of gradual machine learning, the hard instances are gradually labeled in small stages based on the estimated evidential certainty provided by the labeled easier instances. Our extensive experiments on the benchmark datasets have shown that the performance of the proposed approach is considerably better than its unsupervised alternatives, and also highly competitive compared to the state-of-the-art supervised DNN techniques.
Analysis of Automatic Annotation Suggestions for Hard Discourse-Level Tasks in Expert Domains
Schulz, Claudia, Meyer, Christian M., Kiesewetter, Jan, Sailer, Michael, Bauer, Elisabeth, Fischer, Martin R., Fischer, Frank, Gurevych, Iryna
Many complex discourse-level tasks can aid domain experts in their work but require costly expert annotations for data creation. To speed up and ease annotations, we investigate the viability of automatically generated annotation suggestions for such tasks. As an example, we choose a task that is particularly hard for both humans and machines: the segmentation and classification of epistemic activities in diagnostic reasoning texts. We create and publish a new dataset covering two domains and carefully analyse the suggested annotations. We find that suggestions have positive effects on annotation speed and performance, while not introducing noteworthy biases. Envisioning suggestion models that improve with newly annotated texts, we contrast methods for continuous model adjustment and suggest the most effective setup for suggestions in future expert tasks.
Uncertainty-guided Continual Learning with Bayesian Neural Networks
Ebrahimi, Sayna, Elhoseiny, Mohamed, Darrell, Trevor, Rohrbach, Marcus
Continual learning aims to learn new tasks without forgetting previously learned ones. This is especially challenging when one cannot access data from previous tasks and when the model has a fixed capacity. Current regularization-based continual learning algorithms need an external representation and extra computation to measure the parameters' importance. In contrast, we propose Uncertainty-guided Continual Bayesian Neural Networks (UCB), where the learning rate adapts according to the uncertainty defined in the probability distribution of the weights in networks. Uncertainty is a natural way to identify what to remember and what to change as we continually learn, allowing to mitigate catastrophic forgetting. We also show a variant of our model, which uses uncertainty for weight pruning and retains task performance after pruning by saving binary masks per tasks. We evaluate our UCB approach extensively on diverse object classification datasets with short and long sequences of tasks and report superior or on-par performance compared to existing approaches. Additionally, we show that our model does not necessarily need task information at test time, i.e. it does not presume knowledge of which task a sample belongs to.
Meeting on the development of artificial intelligence technologies
Before the meeting, the head of state was told about the academic process at School 21 and had a brief conversation with students. The President was informed about the school by Head of Sberbank German Gref and school Principal Svetlana Infimovskaya. The students of the school can study the following areas: Algorithms, Graphics, Mobile Development, Computer Security, Robot Technology, and Artificial Intelligence to name a few. The school has 940 students today. On average, students are expected to study for 2–3.5 years. The course includes two practical training sessions in relevant companies for six months or more. Today I suggest that we discuss concrete steps that will form the foundation for our National Strategy on the development of artificial intelligence technologies. We have repeatedly spoken about the need for such a comprehensive document. I also mentioned it in this year's Address to the Federal Assembly. This is indeed one of the key areas of technological development that determines and will continue to determine the future of the entire world. The artificial intelligence mechanisms will allow for quick real-time decision-making based on analysing vast amounts of information known as big data, which provides tremendous advantages in terms of quality and performance. In addition, such mechanisms are unparalleled in history in terms of their impact on the economy and productivity, the effectiveness of management, education, healthcare and daily life. However, vying for technological leadership, primarily, in the sphere of artificial intelligence – and you are all very well aware of this, colleagues – has already lead to global competition. New products and solutions are being created at an exponential growth rate. I have said it before and I will say it now: he who can establish a monopoly in artificial intelligence – we are aware of the consequences – will rule the world. It is no accident that many developed countries of the world have already adopted action plans to develop such technologies. Of course, we must ensure technological sovereignty in the realm of artificial intelligence. This is the most important prerequisite for the viability of our businesses and the economy, the quality of life for Russian citizens, security and, finally, our defence capability. Here, we are not just talking about algorithms for addressing individual and highly specialised problems; what we need are universal solutions, the use of which gives the optimum effect in any industry. In order to achieve such an ambitious goal in AI technology, we are objectively positioned to have a good start and we have a serious competitive edge. Today, Russia boasts one of the world's highest penetration rates for mobile communications and internet access, as well as for the development of electronic services.