Multi-Exit models (MEMs) use an early-exit strategy to improve the accuracy and efficiency of deep neural networks (DNNs) by allowing samples to exit the network before the last layer. However, the effectiveness of MEMs in the presence of distribution shifts remains largely unexplored. Our work examines how distribution shifts generated by common image corruptions affect the accuracy/efficiency of MEMs. We find that under common corruptions, early-exiting at the first correct exit reduces the inference cost and provides a significant boost in accuracy ( 10%) over exiting at the last layer. However, with realistic early-exit strategies, which do not assume knowledge about the correct exits, MEMs still reduce inference cost but provide a marginal improvement in accuracy ( 1%) compared to exiting at the last layer. Moreover, the presence of distribution shift widens the gap between an MEM's maximum classification accuracy and realistic early-exit strategies by 5% on average compared with the gap on in-distribution data. Our empirical analysis shows that the lack of calibration due to a distribution shift increases the susceptibility of such early-exit strategies to exit early and increases misclassification rates. Furthermore, the lack of calibration increases the inconsistency in the predictions of the model across exits, leading to both inefficient inference and more misclassifications compared with evaluation on in-distribution data. Finally, we propose two metrics, underthinking and overthinking, that quantify the different behavior of practical early-exit strategy under distribution shifts, and provide insights into improving the practical utility of MEMs.

This paper introduces a scalable algorithmic framework (HyperEF) for spectral coarsening (decomposition) of large-scale hypergraphs by exploiting hyperedge effective resistances. Motivated by the latest theoretical framework for low-resistance-diameter decomposition of simple graphs, HyperEF aims at decomposing large hypergraphs into multiple node clusters with only a few inter-cluster hyperedges. The key component in HyperEF is a nearly-linear time algorithm for estimating hyperedge effective resistances, which allows incorporating the latest diffusion-based non-linear quadratic operators defined on hypergraphs. To achieve good runtime scalability, HyperEF searches within the Krylov subspace (or approximate eigensubspace) for identifying the nearly-optimal vectors for approximating the hyperedge effective resistances. In addition, a node weight propagation scheme for multilevel spectral hypergraph decomposition has been introduced for achieving even greater node coarsening ratios. When compared with state-of-the-art hypergraph partitioning (clustering) methods, extensive experiment results on real-world VLSI designs show that HyperEF can more effectively coarsen (decompose) hypergraphs without losing key structural (spectral) properties of the original hypergraphs, while achieving over $70\times$ runtime speedups over hMetis and $20\times$ speedups over HyperSF.

It has been experimentally demonstrated that humans are able to learn in a manner that allows them to make predictions on categories for which they have not seen any examples (Malaviya et al., 2022). Sucholutsky and Schonlau (2020) have recently presented a machine learning approach that aims to do the same. They utilise synthetically generated data and demonstrate that it is possible to achieve sub-linear scaling and develop models that can learn to recognise N classes from M training samples where M is less than N - aka less-than-one shot learning. Their method was, however, defined for univariate or simple multivariate data (Sucholutsky et al., 2021). We extend it to work on large, high-dimensional and real-world datasets and empirically validate it in this new and challenging setting. We apply this method to learn previously unseen NLP tasks from very few examples (4, 8 or 16). We first generate compact, sophisticated less-than-one shot representations called soft-label prototypes which are fitted on training data, capturing the distribution of different classes across the input domain space. We then use a modified k-Nearest Neighbours classifier to demonstrate that soft-label prototypes can classify data competitively, even outperforming much more computationally complex few-shot learning methods.

As the Internet developed rapidly, it is important to choose suitable web services from a wide range of candidates. Quality of service (QoS) describes the performance of a web service dynamically with respect to the service requested by the service consumer. Moreover, the latent factorization of tenors (LFT) is very effective for discovering temporal patterns in high dimensional and sparse (HiDS) tensors. However, current LFT models suffer from a low convergence rate and rarely account for the effects of outliers. To address the above problems, this paper proposes an Alternating direction method of multipliers (ADMM)-based Outlier-Resilient Nonnegative Latent-factorization of Tensors model. We maintain the non-negativity of the model by constructing an augmented Lagrangian function with the ADMM optimization framework. In addition, the Cauchy function is taken as the metric function to reduce the impact on the model training. The empirical work on two dynamic QoS datasets shows that the proposed method has faster convergence and better performance on prediction accuracy.

Variational Graph Autoencoders (VGAEs) are powerful models for unsupervised learning of node representations from graph data. In this work, we systematically analyze modeling node attributes in VGAEs and show that attribute decoding is important for node representation learning. We further propose a new learning model, interpretable NOde Representation with Attribute Decoding (NORAD). The model encodes node representations in an interpretable approach: node representations capture community structures in the graph and the relationship between communities and node attributes. We further propose a rectifying procedure to refine node representations of isolated notes, improving the quality of these nodes' representations. Our empirical results demonstrate the advantage of the proposed model when learning graph data in an interpretable approach.

A biological system is a complex network of heterogeneous molecular entities and their interactions contributing to various biological characteristics of the system. However, current biological networks are noisy, sparse, and incomplete, limiting our ability to create a holistic view of the biological system and understand the biological phenomena. Experimental identification of such interactions is both time-consuming and expensive. With the recent advancements in high-throughput data generation and significant improvement in computational power, various computational methods have been developed to predict novel interactions in the noisy network. Recently, deep learning methods such as graph neural networks have shown their effectiveness in modeling graph-structured data and achieved good performance in biomedical interaction prediction. However, graph neural networks-based methods require human expertise and experimentation to design the appropriate complexity of the model and significantly impact the performance of the model. Furthermore, deep graph neural networks face overfitting problems and tend to be poorly calibrated with high confidence on incorrect predictions. To address these challenges, we propose Bayesian model selection for graph convolutional networks to jointly infer the most plausible number of graph convolution layers (depth) warranted by data and perform dropout regularization simultaneously. Experiments on four interaction datasets show that our proposed method achieves accurate and calibrated predictions. Our proposed method enables the graph convolutional networks to dynamically adapt their depths to accommodate an increasing number of interactions.

This paper demonstrates how Automated Machine Learning (AutoML) methods can be used as effective surrogate models in engineering design problems. To do so, we consider the challenging problem of structurally-performant bicycle frame design and demonstrate across-the-board dominance by AutoML in regression and classification surrogate modeling tasks. We also introduce FRAMED -- a parametric dataset of 4500 bicycle frames based on bicycles designed by practitioners and enthusiasts worldwide. Accompanying these frame designs, we provide ten structural performance values such as weight, displacements under load, and safety factors computed using finite element simulations for all the bicycle frame designs. We formulate two challenging test problems: a performance-prediction regression problem and a feasibility-prediction classification problem. We then systematically search for optimal surrogate models using Bayesian hyperparameter tuning and neural architecture search. Finally, we show how a state-of-the-art AutoML method can be effective for both regression and classification problems. We demonstrate that the proposed AutoML models outperform the strongest gradient boosting and neural network surrogates identified through Bayesian optimization by an improved F1 score of 24\% for classification and reduced mean absolute error by 12.5\% for regression. Our work introduces a dataset for bicycle design practitioners, provides two benchmark problems for surrogate modeling researchers, and demonstrates the advantages of AutoML in machine learning tasks. The dataset and code are provided at \url{http://decode.mit.edu/projects/framed/}.

Ahmedabad is quickly becoming the top choice for recent graduates seeking to launch their careers in the city's IT and software businesses. With so many top-notch IT companies to choose from, it's surely a terrific spot to get started. It is predicted that in the years to come, demand for AI and associated technologies would grow rapidly. Businesses will employ AI to develop better solutions and to develop specialised internal solutions. Naturally, there will be an increase in demand for AI in the future. It makes sense that one of the most in-demand skills in the technological field, which has impacted practically every industry from automotive, manufacturing, and healthcare to cybersecurity, software, and IoT, is AI.

This article was published as a part of the Data Science Blogathon. Hey, are you working on a data science project, solving a problem statement related to data science, or experimenting with a statistical test to make further decisions and handling the most repeatedly cited statistical term, 'correlation'? Willing to correctly interpret these statistical terms (covariance, parametric and non-parametric correlation)? Which method will outperform or underperform in your problem statement in a specific scenario (i.e., sensitive to outliers, type of distribution, etc.)? Working on sample data and willing to know the probable error of the coefficient?

This article was published as a part of the Data Science Blogathon. Almost every data scientist must have encountered the data for which they need to perform imbalanced binary classification. Imbalanced data means the number of rows or frequency of data points of one class is much more than the other class. In other words, the ratio of the value counts of classes is much higher. Such data set is known as an imbalanced dataset in which the class having more data points is the majority class and the other is the minority class. The imbalance makes the classification more challenging.