Goto

Collaborating Authors

 error


Knowledge Distillation by On-the-Fly Native Ensemble

xu lan, Xiatian Zhu, Shaogang Gong

Neural Information Processing Systems

Knowledge distillation is effective to train the small and generalisable network models for meeting the low-memory and fast running requirements. Existing offline distillation methods rely on a strong pre-trained teacher, which enables favourable knowledge discovery and transfer but requires a complex two-phase training procedure. Online counterparts address this limitation at the price of lacking a high-capacity teacher. In this work, we present an On-the-fly Native Ensemble (ONE) learning strategyforone-stage online distillation.


Bootstrapping the Error of Oja's Algorithm

Neural Information Processing Systems

We consider the problem of quantifying uncertainty for the estimation error of the leading eigenvector from Oja's algorithm for streaming principal component analysis, where the data are generated IID from some unknown distribution. By combining classical tools from the U-statistics literature with recent results on high-dimensional central limit theorems for quadratic forms of random vectors and concentration of matrix products, we establish a weighted $\chi^2$ approximation result for the $\sin^2$ error between the population eigenvector and the output of Oja's algorithm. Since estimating the covariance matrix associated with the approximating distribution requires knowledge of unknown model parameters, we propose a multiplier bootstrap algorithm that may be updated in an online manner. We establish conditions under which the bootstrap distribution is close to the corresponding sampling distribution with high probability, thereby establishing the bootstrap as a consistent inferential method in an appropriate asymptotic regime.


Sample Compression for Continual Learning

Comeau, Jacob, Bazinet, Mathieu, Germain, Pascal, Subakan, Cem

arXiv.org Artificial Intelligence

Continual learning algorithms aim to learn from a sequence of tasks, making the training distribution non-stationary. The majority of existing continual learning approaches in the literature rely on heuristics and do not provide learning guarantees for the continual learning setup. In this paper, we present a new method called 'Continual Pick-to-Learn' (CoP2L), which is able to retain the most representative samples for each task in an efficient way. The algorithm is adapted from the Pick-to-Learn algorithm, rooted in the sample compression theory. This allows us to provide high-confidence upper bounds on the generalization loss of the learned predictors, numerically computable after every update of the learned model. We also empirically show on several standard continual learning benchmarks that our algorithm is able to outperform standard experience replay, significantly mitigating catastrophic forgetting.


HalluVerse25: Fine-grained Multilingual Benchmark Dataset for LLM Hallucinations

Abdaljalil, Samir, Kurban, Hasan, Serpedin, Erchin

arXiv.org Artificial Intelligence

Large Language Models (LLMs) are increasingly used in various contexts, yet remain prone to generating non-factual content, commonly referred to as "hallucinations". The literature categorizes hallucinations into several types, including entity-level, relation-level, and sentence-level hallucinations. However, existing hallucination datasets often fail to capture fine-grained hallucinations in multilingual settings. In this work, we introduce HalluVerse25, a multilingual LLM hallucination dataset that categorizes fine-grained hallucinations in English, Arabic, and Turkish. Our dataset construction pipeline uses an LLM to inject hallucinations into factual biographical sentences, followed by a rigorous human annotation process to ensure data quality. We evaluate several LLMs on HalluVerse25, providing valuable insights into how proprietary models perform in detecting LLM-generated hallucinations across different contexts.


GroMo: Plant Growth Modeling with Multiview Images

Bhatt, Ruchi, Bansal, Shreya, Chander, Amanpreet, Kaur, Rupinder, Singh, Malya, Kankanhalli, Mohan, Saddik, Abdulmotaleb El, Saini, Mukesh Kumar

arXiv.org Artificial Intelligence

Understanding plant growth dynamics is essential for applications in agriculture and plant phenotyping. We present the Growth Modelling (GroMo) challenge, which is designed for two primary tasks: (1) plant age prediction and (2) leaf count estimation, both essential for crop monitoring and precision agriculture. For this challenge, we introduce GroMo25, a dataset with images of four crops: radish, okra, wheat, and mustard. Each crop consists of multiple plants (p1, p2, ..., pn) captured over different days (d1, d2, ..., dm) and categorized into five levels (L1, L2, L3, L4, L5). Each plant is captured from 24 different angles with a 15-degree gap between images. Participants are required to perform both tasks for all four crops with these multiview images. We proposed a Multiview Vision Transformer (MVVT) model for the GroMo challenge and evaluated the crop-wise performance on GroMo25. MVVT reports an average MAE of 7.74 for age prediction and an MAE of 5.52 for leaf count. The GroMo Challenge aims to advance plant phenotyping research by encouraging innovative solutions for tracking and predicting plant growth. The GitHub repository is publicly available at https://github.com/mriglab/GroMo-Plant-Growth-Modeling-with-Multiview-Images.


Comparative Analysis of Shear Strength Prediction Models for Reinforced Concrete Slab-Column Connections

Wahab, Sarmed, Mahmoudabadi, Nasim Shakouri, Waqas, Sarmad, Herl, Nouman, Iqbal, Muhammad, Alam, Khurshid, Ahmad, Afaq

arXiv.org Artificial Intelligence

This research aims at comparative analysis of shear strength prediction at slab-column connection, unifying machine learning, design codes and Finite Element Analysis. Current design codes (CDCs) of ACI 318-19 (ACI), Eurocode 2 (EC2), Compressive Force Path (CFP) method, Feed Forward Neural Network (FNN) based Artificial Neural Network (ANN), PSO-based FNN (PSOFNN), and BAT algorithm-based BATFNN are used. The study is complemented with FEA of slab for validating the experimental results and machine learning predictions.In the case of hybrid models of PSOFNN and BATFNN, mean square error is used as an objective function to obtain the optimized values of the weights, that are used by Feed Forward Neural Network to perform predictions on the slab data. Seven different models of PSOFNN, BATFNN, and FNN are trained on this data and the results exhibited that PSOFNN is the best model overall. PSOFNN has the best results for SCS=1 with highest value of R as 99.37% and lowest of MSE, and MAE values of 0.0275%, and 1.214% respectively which are better than the best FNN model for SCS=4 having the values of R, MSE, and MAE as 97.464%, 0.0492%, and 1.43%, respectively.


Tensorflow Plugin - Metal - Apple Developer

#artificialintelligence

Error: "Could not find a version that satisfies the requirement tensorflow-macos (from versions: none)." A tensorflow installation wheel that matches the current Python environment couldn't be found by the package manager. Check that the Python version used in the environment is supported (Python 3.8, Python 3.9, Python 3.10). Complex data type isn't supported by tensorflow-metal. Error: "Cannot assign a device for operation: Could not satisfy explicit device specification because the node was colocated with a group of nodes that required incompatible device."


Intellectual abilities of artificial intelligence (AI) - Semiwiki

#artificialintelligence

To understand AI’s capabilities and abilities we need to recognize the different components and subsets of AI. Terms like Neural Networks, Machine Learning (ML), and Deep Learning, need to be define and explained. In general, Artificial intelligence (AI) refers to the simulation of human intelligence in machines that are programmed to think like humans and…


cleanlab 2.0: Automatically Find Errors in ML Datasets

#artificialintelligence

Distributed ML is an active area of work, in both academia and industry, and it has been for some time now. Companies like Google were doing distributed machine learning decades ago. For some use cases, libraries like scikit-learn are totally adequate, while for other use cases, e.g. when using sophisticated models that require a lot of compute to train, training over large datasets that don't fit on a single node, distributed computing is essential. On the topic of data storage: in some cases, system builders do co-design the data storage and data processing, e.g. Such co-design can give performance gains.


Loss Functions: An Explainer - KDnuggets

#artificialintelligence

Loss function is a method that evaluates how well the algorithm learns the data and produces correct outputs. It computes the distance between our predicted value and the actual value using a mathematical formula. In layman's terms, a loss function measures how wrong the model is in terms of its ability to estimate the relationship between x and y. Below is a list of types of loss functions for both Classification and Regression tasks. Cross Entropy and Log Loss measure the same thing, however they are not the same and is used for Classification tasks.