Goto

Collaborating Authors

 Statistical Learning


Beyond Linear Regression

#artificialintelligence

Linear regression is among the primary/entry-level Machine Learning (ML) models. It's not even wrong to say that it's the synonym of the "Hello world" program for Data scientists. Finding the linear regression coefficients β_1, …, β_p involves finding the "best" linear combination of variables that approaches the response. Said differently, finding the coefficients that minimize the mean squared error (MSE). It's possible to endow the regression coefficients with some extra properties by considering the MSE plus an additional penalty term.


Geometric deep learning for computational mechanics Part II: Graph embedding for interpretable multiscale plasticity

arXiv.org Artificial Intelligence

The composition of a macroscopic plasticity model often requires the following steps. First, there are observations of causality relations deduced by modelers to hypothesize mechanisms that lead to the plastic flow. These causality relations along with constraints inferred from physics and universally accepted principles lead to mathematical equations. For instance, the family of Gurson models employs the observation of void growth to employ the yield surface (Gurson, 1977). Crystal plasticity models relate the plastic flow with slip systems to predict the anisotropic responses of single crystals (Rice, 1971; Uchic et al., 2004; Clayton, 2010; Ma and Sun, 2020; Ma et al., 2021). Granular plasticity models propose theories that relate the fabric of force chains and porosity to the onset of plastic yielding and the resultant plastic flow (Cowin, 1985; Kuhn et al., 2015; Wang and Sun, 2018; Sun et al., 2022). Finally, the mathematical equations are then either used directly in engineering analysis and designs (e.g. the Mohr-Coulomb envelope) or are incorporated into a boundary value problem in which the approximation solution can be obtained from a partial differential equation solver that provides incremental updates of stress-strain relations. However, a subtle but significant limitation of this paradigm is that it imposes the burdens on modelers of being able to describe the mechanisms verbally via terminologies or atomic facts (cf.


Automatically Categorising GitHub Repositories by Application Domain

arXiv.org Artificial Intelligence

For example, there are limited means available to separate repositories containing engineered software projects from other repositories, such as personal projects or those that use GitHub for free cloud storage (Kalliamvakou et al., 2014; Munaiah et al., 2017). To make it easier for users to identify relevant repositories for their wide variety of use cases, GitHub has been adding features to its service, such as README files, topics tags, and showcases (where contributors describe, add keywords, and label their repository). However, these features are insufficient for many use cases. For example, while achieving generalizability of the results is the primary objective of many empirical papers, modern computing research is largely application domain independent (Capiluppi et al., 2020). Application domains are the sections of reality for which a software system is designed. Their importance relies on their serving as the starting point for actual state analysis and usually includes domain-specific language, meaning that developers in this domain think about their project in a specific way, with particular terms and concepts (Züllighoven, 2004). Application domains are not a feature currently implemented by GitHub to catalogue repositories. Previous work has found that repository quality indicators, such as object-oriented metrics, can be "extremely sensitive to application domains" (Capiluppi and Ajienka, 2019), and that the application domain is an important factor in predicting repository popularity (Borges et al., 2016). Furthermore, since documentation of GitHub repositories is often incomplete (Prana et al., 2019), information about the application domain of a repository can be crucial to gain a high-level understanding of its content and purpose.


On Connecting Deep Trigonometric Networks with Deep Gaussian Processes: Covariance, Expressivity, and Neural Tangent Kernel

arXiv.org Artificial Intelligence

Deep Gaussian Process (DGP) as a model prior in Bayesian learning intuitively exploits the expressive power in function composition. DGPs also offer diverse modeling capabilities, but inference is challenging because marginalization in latent function space is not tractable. With Bochner's theorem, DGP with squared exponential kernel can be viewed as a deep trigonometric network consisting of the random feature layers, sine and cosine activation units, and random weight layers. In the wide limit with a bottleneck, we show that the weight space view yields the same effective covariance functions which were obtained previously in function space. Also, varying the prior distributions over network parameters is equivalent to employing different kernels. As such, DGPs can be translated into the deep bottlenecked trig networks, with which the exact maximum a posteriori estimation can be obtained. Interestingly, the network representation enables the study of DGP's neural tangent kernel, which may also reveal the mean of the intractable predictive distribution. Statistically, unlike the shallow networks, deep networks of finite width have covariance deviating from the limiting kernel, and the inner and outer widths may play different roles in feature learning. Numerical simulations are present to support our findings.


A Small Survey On Event Detection Using Twitter

arXiv.org Artificial Intelligence

This is evident from popular phenomena such as effects of fake news and online social movements. However the the data obtained from social media presents itself with large volume and velocity, accompanied by significant amount of irrelevant data pertaining to general discussions, personal messages and spam. Social media has been shown to be effective for detecting, forecasting and tracking real world events. The ability to detect real world events is crucial and has applications in disease surveillance, commerce, governance and other areas. Thus extraction of useful information and modelling the characteristics of social media to detect real world events is an important problem. 2 RESEARCH PROBLEM To outline the research problem we need to define events, which has multiple interpretations.


Streaming Algorithms for Diversity Maximization with Fairness Constraints

arXiv.org Artificial Intelligence

Diversity maximization is a fundamental problem with wide applications in data summarization, web search, and recommender systems. Given a set $X$ of $n$ elements, it asks to select a subset $S$ of $k \ll n$ elements with maximum \emph{diversity}, as quantified by the dissimilarities among the elements in $S$. In this paper, we focus on the diversity maximization problem with fairness constraints in the streaming setting. Specifically, we consider the max-min diversity objective, which selects a subset $S$ that maximizes the minimum distance (dissimilarity) between any pair of distinct elements within it. Assuming that the set $X$ is partitioned into $m$ disjoint groups by some sensitive attribute, e.g., sex or race, ensuring \emph{fairness} requires that the selected subset $S$ contains $k_i$ elements from each group $i \in [1,m]$. A streaming algorithm should process $X$ sequentially in one pass and return a subset with maximum \emph{diversity} while guaranteeing the fairness constraint. Although diversity maximization has been extensively studied, the only known algorithms that can work with the max-min diversity objective and fairness constraints are very inefficient for data streams. Since diversity maximization is NP-hard in general, we propose two approximation algorithms for fair diversity maximization in data streams, the first of which is $\frac{1-\varepsilon}{4}$-approximate and specific for $m=2$, where $\varepsilon \in (0,1)$, and the second of which achieves a $\frac{1-\varepsilon}{3m+2}$-approximation for an arbitrary $m$. Experimental results on real-world and synthetic datasets show that both algorithms provide solutions of comparable quality to the state-of-the-art algorithms while running several orders of magnitude faster in the streaming setting.


HPO X ELA: Investigating Hyperparameter Optimization Landscapes by Means of Exploratory Landscape Analysis

arXiv.org Artificial Intelligence

Hyperparameter optimization (HPO) is a key component of machine learning models for achieving peak predictive performance. While numerous methods and algorithms for HPO have been proposed over the last years, little progress has been made in illuminating and examining the actual structure of these black-box optimization problems. Exploratory landscape analysis (ELA) subsumes a set of techniques that can be used to gain knowledge about properties of unknown optimization problems. In this paper, we evaluate the performance of five different black-box optimizers on 30 HPO problems, which consist of two-, three- and five-dimensional continuous search spaces of the XGBoost learner trained on 10 different data sets. This is contrasted with the performance of the same optimizers evaluated on 360 problem instances from the black-box optimization benchmark (BBOB). We then compute ELA features on the HPO and BBOB problems and examine similarities and differences. A cluster analysis of the HPO and BBOB problems in ELA feature space allows us to identify how the HPO problems compare to the BBOB problems on a structural meta-level. We identify a subset of BBOB problems that are close to the HPO problems in ELA feature space and show that optimizer performance is comparably similar on these two sets of benchmark problems. We highlight open challenges of ELA for HPO and discuss potential directions of future research and applications.


ANOVA-based Automatic Attribute Selection and a Predictive Model for Heart Disease Prognosis

arXiv.org Artificial Intelligence

Studies show that Studies that cardiovascular diseases (CVDs) are malignant for human health. Thus, it is important to have an efficient way of CVD prognosis. In response to this, the healthcare industry has adopted machine learning-based smart solutions to alleviate the manual process of CVD prognosis. Thus, this work proposes an information fusion technique that combines key attributes of a person through analysis of variance (ANOVA) and domain experts' knowledge. It also introduces a new collection of CVD data samples for emerging research. There are thirty-eight experiments conducted exhaustively to verify the performance of the proposed framework on four publicly available benchmark datasets and the newly created dataset in this work. The ablation study shows that the proposed approach can achieve a competitive mean average accuracy (mAA) of 99.2% and a mean average AUC of 97.9%.


June 2022: "Top 40" New CRAN Packages

#artificialintelligence

One hundred eighty-nine new packages made it to CRAN in June. Here are my “Top 40” selections in eleven categories: Computational Methods, Data, Ecology, Genomics, Machine Learning, Mathematics, Medicine, Statistics, Time Series, Utilities, and Visualizations. Computational Methods itp v1.2.0: Implements the interpolate, truncate, project root-finding algorithm developed by Oliveira & Takahashi (2021). The vignette provides an overview. QR v0..1.3: Provides a function to perform QR factorization without pivoting to a real or complex matrix. It is based on LAPACK. See the vignette. qsplines v1.0.0: Provides functions to create quaterion splines. See Barry & Goldman (1988) and Kochanek & Bartels (1984) for the details and look here for an example. VMDecomp v1.0.1: Implements the variational mode decomposition and two-dimensional variational mode decomposition algorithm. See Dragomiretskiy & Zosso (2014) for background and the vignette for examples. Data cmch v0.2.0: Implements a wrapper around the Canadian Mortgage and Housing Corporation web interface and enables programmatic and reproducible access to a wide variety of housing data. See the vignette for examples. EDIutils v1.0.1: Implements a client for the Environmental Data Initiative repository REST API and provides access to ecological data and metadata. There are five short vignettes: Evaluate & upload, Citation Metrics, Download Metrics, Search andaccess, and Tests. globaltrends v0.0.12: Provides functions to access global search volumes from the Google Trends portal. This working paper outlines the package’s methodological foundations and potential applications. See the vignette to get started. kaigiroku v0.5: Allows users to search and download data from the API for Japanese Diet proceedings. Look here for examples. NasdaqDataLink v1.0.0: Provides functions to interact directly with the Nasdaq Data Link API and obtain data in a number of formats. Look here for API documentation and here for package information. stortingscrape v0.1.1: Provides functions for retrieving data from the Norwegian Parliament, through the Norwegian Parliament API. See the vingette for an introduction. Ecology PointedSDMs v1.0.6: Provides tools to build integrated species distribution models and includes tools to run spatial cross-validation and plotting. See Issac et al. (2020) for and introduction to the methods. There is a Setophaga Example and an example for the Solitary Tinamou. restoptr v1.0.1: Implements a flexible framework for ecological restoration planning that aims to identify priority areas for restoration efforts using optimization algorithms described in Justeau-Allaire et al. 2021. See the vignette to get started. Genomics scapGNN v0.1.1: Implements a single cell active pathway analysis tool based on the graph neural network algorithm described in Scarselli et al. (2009) and Kipf & Welling (2017). This may be used to construct a gene-cell association network, infer pathway activity scores from different single cell modalities data and more. See the vignette for an overview and examples. SRTsim v0.99.2: Implements an independent, reproducible, and flexible Spatially Resolved Transcriptomics simulation framework that can be used to facilitate the development analytical methods and for a wide variety of SRT-specific analyses. See the vignette. xQTLbiolinks v1.1.1: Implements tools to query, download, and visualize of molecular quantitative trait locus and gene expression data from public resources through the GTEx API. There is a Quick Start Guide and vignettes on Colocalization, Specivicity, and Visualization. Machine Learning agua v0.0.1: Enables users to specify h2o as an engine for several tidymodels modeling methods. See README for examples. MagmaClustR V1.0.0: Implements two main algorithms, called Magma (Leroy et al. (2022) and MagmaClust (Leroy et al. (2020)), using a multi-task Gaussian processes (GP) model to perform predictions for supervised learning problems. See README for examples. openai v0.1.0: Provides a wrapper for OpenAI API endpoints including engines, completions, edits, files, fine-tunes, embeddings and legacy searches, classifications, and answers endpoints. See README to get started. sketching v0.1.0: Provides functions to construct sketches of data via random subspace embeddings. See Lee & Ng (2022) for the theory and the vignette for examples. webmorphR v0..1.1: Provides functions to create reproducible image stimuli, specialised for face images with psychomorph or webmorph templates. See README to get started. Mathematics GeneralizedWendland v0.5-2: Implements the fully parameterized generalized Wendland covariance function for use in Gaussian process models, as well as multiple methods for approximating it via covariance interpolation. The available methods are linear interpolation, polynomial interpolation, and cubic spline interpolation. See Bevilacqua et al. (2022) and the vignette for examples. jacobi v2.0.0: Evaluates Jacobi theta functions and related functions including the Weierstrass elliptic function, the Weierstrass sigma function, the Weierstrass zeta function, the Klein j-function, the Dedekind eta function, the lambda modular function, Jacobi elliptic functions, Neville theta functions, and the Eisenstein series for real and complex variables. Look here for some images. Medicine clinicalsignificance v1.0.0: Implements the clinical significance algorithm proposed by Jacobson et al. (1984) to determine if an intervention has a meaningful practical effect. There is a Getting Started Guide and vignettes on Cutoffs and Plots. PlatformDesign v1.0.1: Provides functions to calculate design parameters for an optimal two-period, multi-arm platform design allowing pre-planned deferred arms to be added during the trial. See Dunnett (1955) for background and the vignette for some theory and examples. Statistics bayesassurance v0.1.0: Provides functions to compute Bayesian assurance under various settings characterized by different assumptions and objectives, including precision-based conditions, credible intervals, and goal functions. See Pan & Banerjee (2021) for the theory. There are vignettes for using closed form solutions, the conjugate linear model, and precision based conditions. DSSP v0.1.1: Provides functions to draw samples from the direct sampling spatial prior model as described in White, Sun, & Speckman (2019). See the vignette for examples. edibble v0.1.0: Implements a system to facilitate designing comparative experiments using the grammar of experimental designs. See the edibble-book for documentation. mixgb v0.1.0: Implements a method for multiple imputation using XGBoost, bootstrapping and predictive mean matching as described in Deng and Lumley (2021). There is an Introduction and a vignette on Imputing new data with a saved imputer. outerbase v0.1.0: Implements in new method for high-dimensional regression using outer product models. See Plumlee (2014) and Plumlee et al. (2021) for background. There is a Getting started guide, a Base walkthrough, and vignettes on Learning from data and Speeding up inference. PFIM v5.0: Provides functions to evaluate or optimize designs for nonlinear mixed effects models using the Fisher Information matrix. See Malle & Baccar D (1997) and Retout et al. (2007) for background and the vignettes Design evaluation and optimixation (01), Design evaluation and optimixation (02), and Library of models for examples. VirtualPop v1.0.2: Provides functions to generate lifespans and fertility histories in continuous time using individual-level state transition (multi-state) models and data. See the vignettes on Simulation of life histories, Sampling from waiting time distributions, Simulation of individual fertility careers, and Validation. Time Series kssa v0.0.1: Implements the known sub-sequence algorithm described in Benavides et al. (2022), which helps to automatically identify and validate the best method for missing data imputation in a time series. Look here for examples. ts2net v0.1.0: Implements methods to transform time series into networks, a technique which may be useful for complex systems modeling, time series data mining, or time series analysis using networks. For an introduction to the topic and descriptions of the methods see Mitchell (2006), Silva & Zhao (2016), and Silva et al. (2021). See README to get started. Utilities cppchedkR Allows users to run Cppcheck on C/C++ files as an R command or an RStudio addin. See README. . gtExtras v0.4.1: Provides additional functions for creating tables with gt. See README for examples. . Visualization ggpie v0.2.2: Provides functions for creating pie, donut and rose pie plots with ggplot2. See the vignette. ggtrace v0.2.0: Provides ggplot2 geoms that allow groups of data points to be outlined or highlighted for emphasis. See the vignettes Trace lines and Trace points. Morphoscape v1.0.0: Implements adaptive landscape methods first described by Polly et al. (2016) for the integration, analysis and visualization of biological trait data on a phenotypic morphospace which are typically defined by shape metrics. See the vignette. r3js v0.0.1: Provides R and JavaScript functions to allow WebGL-based 3D plotting using the three.js library. See the vignettes: Getting Started, Creating a plot from scratch, and Grouping plot elements. rgl2gltf v1.0.0: Provides functions to work with glTF files which are used to describe 3D models. See the vignette for examples.. . shapviz v0.2.0: Provides functions to visualize SHapley Additive exPlanations (SHAP), such as waterfall plots, force plots, various types of importance plots, and dependence plots. See Lundberg & Lee (2017) for background and the vignette for examples.


How to use Quantum Machine Learning in your AI projects

#artificialintelligence

With the recent boom in quantum computing research, one can only wonder what drives such attention towards this new field. Classical computers have erased the boundaries between imagination and reality, they have enabled society to communicate with just the touch of a button (or should I say screen), improved the lives of millions of people by automating tough and repetitive tasks, and brought worlds to life in the form of videogames and virtual realities like the upcoming Metaverse. However, none of this could have been possible if it was not for tedious research that led to the constant improvement of computation time and resource allocation. This is the key to the promise of quantum computers. Binge words like Quantum Advantage and Quantum Speedups are self-explanatory, quantum computers could demonstrate an advantage against classical computers in several problems and even perform calculations in almost no time at all.