Statistical Learning
Exception Handling Concepts in Python
In this article, we will discuss error handling in python with a try, except and finally keywords to handle file and data management. Exception handling is a very realistic approach to handle the exception when something wrong happens in the program, in that case, the interpreter stops and raises an exception until it is handled. To handle these types of exceptions in the program for continuing the execution we learn exception handling. Here, we get the output "2" because the print statement is executed. We can also print another statement to check the whole program is executed properly.
Hybrid and Generalized Bayesian Cram\'{e}r-Rao Inequalities via Information Geometry
Mishra, Kumar Vijay, Kumar, M. Ashok
Information geometry is the study of statistical models from a Riemannian geometric point of view. The Fisher information matrix plays the role of a Riemannian metric in this framework. This tool helps us obtain Cram\'{e}r-Rao lower bound (CRLB). This chapter summarizes the recent results which extend this framework to more general Cram\'{e}r-Rao inequalities. We apply Eguchi's theory to a generalized form of Czsisz\'ar $f$-divergence to obtain a Riemannian metric that, at once, is used to obtain deterministic CRLB, Bayesian CRLB, and their generalizations.
Assessment of machine learning methods for state-to-state approaches
Campoli, Lorenzo, Kustova, Elena, Maltseva, Polina
It is well known that numerical simulations of high-speed reacting flows, in the framework of state-to-state formulations, are the most detailed but also often prohibitively computationally expensive. In this work, we start to investigate the possibilities offered by the use of machine learning methods for state-to-state approaches to alleviate such burden. In this regard, several tasks have been identified. Firstly, we assessed the potential of state-of-the-art data-driven regression models based on machine learning to predict the relaxation source terms which appear in the right-hand side of the state-to-state Euler system of equations for a one-dimensional reacting flow of a N$_2$/N binary mixture behind a plane shock wave. It is found that, by appropriately choosing the regressor and opportunely tuning its hyperparameters, it is possible to achieve accurate predictions compared to the full-scale state-to-state simulation in significantly shorter times. Secondly, we investigated different strategies to speed-up our in-house state-to-state solver by coupling it with the best-performing pre-trained machine learning algorithm. The embedding of machine learning methods into ordinary differential equations solvers may offer a speed-up of several orders of magnitude but some care should be paid for how and where such coupling is realized. Performances are found to be strongly dependent on the mutual nature of the interfaced codes. Finally, we aimed at inferring the full solution of the state-to-state Euler system of equations by means of a deep neural network completely by-passing the use of the state-to-state solver while relying only on data. Promising results suggest that deep neural networks appear to be a viable technology also for these tasks.
Artificial intelligence, human rights, democracy, and the rule of law: a primer
Leslie, David, Burr, Christopher, Aitken, Mhairi, Cowls, Josh, Katell, Michael, Briggs, Morgan
In September 2019, the Council of Europe's Committee of Ministers adopted the terms of reference for the Ad Hoc Committee on Artificial Intelligence (CAHAI). The CAHAI is charged with examining the feasibility and potential elements of a legal framework for the design, development, and deployment of AI systems that accord with Council of Europe standards across the interrelated areas of human rights, democracy, and the rule of law. As a first and necessary step in carrying out this responsibility, the CAHAI's Feasibility Study, adopted by its plenary in December 2020, has explored options for an international legal response that fills existing gaps in legislation and tailors the use of binding and non-binding legal instruments to the specific risks and opportunities presented by AI systems. The Study examines how the fundamental rights and freedoms that are already codified in international human rights law can be used as the basis for such a legal framework. The purpose of this primer is to introduce the main concepts and principles presented in the CAHAI's Feasibility Study for a general, non-technical audience. It also aims to provide some background information on the areas of AI innovation, human rights law, technology policy, and compliance mechanisms covered therein. In keeping with the Council of Europe's commitment to broad multi-stakeholder consultations, outreach, and engagement, this primer has been designed to help facilitate the meaningful and informed participation of an inclusive group of stakeholders as the CAHAI seeks feedback and guidance regarding the essential issues raised by the Feasibility Study.
Exponential Reduction in Sample Complexity with Learning of Ising Model Dynamics
Dutt, Arkopal, Lokhov, Andrey Y., Vuffray, Marc, Misra, Sidhant
The usual setting for learning the structure and parameters of a graphical model assumes the availability of independent samples produced from the corresponding multivariate probability distribution. However, for many models the mixing time of the respective Markov chain can be very large and i.i.d. samples may not be obtained. We study the problem of reconstructing binary graphical models from correlated samples produced by a dynamical process, which is natural in many applications. We analyze the sample complexity of two estimators that are based on the interaction screening objective and the conditional likelihood loss. We observe that for samples coming from a dynamical process far from equilibrium, the sample complexity reduces exponentially compared to a dynamical process that mixes quickly.
Neurons learn slower than they think
Recent studies revealed complex convergence dynamics in gradient-based methods, which has been little understood so far. Changing the step size to balance between high convergence rate and small generalization error may not be sufficient: maximizing the test accuracy usually requires a larger learning rate than minimizing the training loss. To explore the dynamic bounds of convergence rate, this study introduces \textit{differential capability} into an optimization process, which measures whether the test accuracy increases as fast as a model approaches the decision boundary in a classification problem. The convergence analysis showed that: 1) a higher convergence rate leads to slower capability growth; 2) a lower convergence rate results in faster capability growth and decay; 3) regulating a convergence rate in either direction reduces differential capability.
Topological Regularization for Graph Neural Networks Augmentation
Song, Rui, Giunchiglia, Fausto, Zhao, Ke, Xu, Hao
The complexity and non-Euclidean structure of graph data hinder the development of data augmentation methods similar to those in computer vision. In this paper, we propose a feature augmentation method for graph nodes based on topological regularization, in which topological structure information is introduced into end-to-end model. Specifically, we first obtain topology embedding of nodes through unsupervised representation learning method based on random walk. Then, the topological embedding as additional features and the original node features are input into a dual graph neural network for propagation, and two different high-order neighborhood representations of nodes are obtained. On this basis, we propose a regularization technique to bridge the differences between the two different node representations, eliminate the adverse effects caused by the topological features of graphs directly used, and greatly improve the performance. We have carried out extensive experiments on a large number of datasets to prove the effectiveness of our model.
Permutation-Invariant Subgraph Discovery
Mall, Raghvendra, Parambath, Shameem A., Yufei, Han, Yu, Ting, Chawla, Sanjay
We introduce Permutation and Structured Perturbation Inference (PSPI), a new problem formulation that abstracts many graph matching tasks that arise in systems biology. PSPI can be viewed as a robust formulation of the permutation inference or graph matching, where the objective is to find a permutation between two graphs under the assumption that a set of edges may have undergone a perturbation due to an underlying cause. For example, suppose there are two gene regulatory networks X and Y from a diseased and normal tissue respectively. Then, the PSPI problem can be used to detect if there has been a structural change between the two networks which can serve as a signature of the disease. Besides the new problem formulation, we propose an ADMM algorithm (STEPD) to solve a relaxed version of the PSPI problem. An extensive case study on comparative gene regulatory networks (GRNs) is used to demonstrate that STEPD is able to accurately infer structured perturbations and thus provides a tool for computational biologists to identify novel prognostic signatures. A spectral analysis confirms that STEPD can recover small clique-like perturbations making it a useful tool for detecting permutation-invariant changes in graphs.
A Comparison of Similarity Based Instance Selection Methods for Cross Project Defect Prediction
Hosseini, Seyedrebvar, Turhan, Burak
Context: Previous studies have shown that training data instance selection based on nearest neighborhood (NN) information can lead to better performance in cross project defect prediction (CPDP) by reducing heterogeneity in training datasets. However, neighborhood calculation is computationally expensive and approximate methods such as Locality Sensitive Hashing (LSH) can be as effective as exact methods. Aim: We aim at comparing instance selection methods for CPDP, namely LSH, NN-filter, and Genetic Instance Selection (GIS). Method: We conduct experiments with five base learners, optimizing their hyper parameters, on 13 datasets from PROMISE repository in order to compare the performance of LSH with benchmark instance selection methods NN-Filter and GIS. Results: The statistical tests show six distinct groups for F-measure performance. The top two group contains only LSH and GIS benchmarks whereas the bottom two groups contain only NN-Filter variants. LSH and GIS favor recall more than precision. In fact, for precision performance only three significantly distinct groups are detected by the tests where the top group is comprised of NN-Filter variants only. Recall wise, 16 different groups are identified where the top three groups contain only LSH methods, four of the next six are GIS only and the bottom five contain only NN-Filter. Finally, NN-Filter benchmarks never outperform the LSH counterparts with the same base learner, tuned or non-tuned. Further, they never even belong to the same rank group, meaning that LSH is always significantly better than NN-Filter with the same learner and settings. Conclusions: The increase in performance and the decrease in computational overhead and runtime make LSH a promising approach. However, the performance of LSH is based on high recall and in environments where precision is considered more important NN-Filter should be considered.
The CSO Classifier: Ontology-Driven Detection of Research Topics in Scholarly Articles
Salatino, Angelo A., Osborne, Francesco, Thanapalasingam, Thiviyan, Motta, Enrico
Classifying research papers according to their research topics is an important task to improve their retrievability, assist the creation of smart analytics, and support a variety of approaches for analysing and making sense of the research environment. In this paper, we present the CSO Classifier, a new unsupervised approach for automatically classifying research papers according to the Computer Science Ontology (CSO), a comprehensive ontology of re-search areas in the field of Computer Science. The CSO Classifier takes as input the metadata associated with a research paper (title, abstract, keywords) and returns a selection of research concepts drawn from the ontology. The approach was evaluated on a gold standard of manually annotated articles yielding a significant improvement over alternative methods.