Statistical Learning
Classifying YouTube Comments Based on Sentiment and Type of Sentence
Pokharel, Rhitabrat, Bhatta, Dixit
As a YouTube channel grows, each video can potentially collect enormous amounts of comments that provide direct feedback from the viewers. These comments are a major means of understanding viewer expectations and improving channel engagement. However, the comments only represent a general collection of user opinions about the channel and the content. Many comments are poorly constructed, trivial, and have improper spellings and grammatical errors. As a result, it is a tedious job to identify the comments that best interest the content creators. In this paper, we extract and classify the raw comments into different categories based on both sentiment and sentence types that will help YouTubers find relevant comments for growing their viewership. Existing studies have focused either on sentiment analysis (positive and negative) or classification of sub-types within the same sentence types (e.g., types of questions) on a text corpus. These have limited application on non-traditional text corpus like YouTube comments. We address this challenge of text extraction and classification from YouTube comments using well-known statistical measures and machine learning models. We evaluate each combination of statistical measure and the machine learning model using cross validation and $F_1$ scores. The results show that our approach that incorporates conventional methods performs well on the classification task, validating its potential in assisting content creators increase viewer engagement on their channel.
Efficient, Anytime Algorithms for Calibration with Isotonic Regression under Strictly Convex Losses
We investigate the calibration of estimations to increase performance with an optimal monotone transform on the estimator outputs. We start by studying the traditional square error setting with its weighted variant and show that the optimal monotone transform is in the form of a unique staircase function. We further show that this staircase behavior is preserved for general strictly convex loss functions. Their optimal monotone transforms are also unique, i.e., there exist a single staircase transform that achieves the minimum loss. We propose a linear time and space algorithm that can find such optimal transforms for specific loss settings. Our algorithm has an online implementation where the optimal transform for the samples observed so far are found in linear space and amortized time when the samples arrive in an ordered fashion. We also extend our results to cases where the functions are not trivial to individually optimize and propose an anytime algorithm, which has linear space and pseudo-linearithmic time complexity.
Predicting Cancer Using Supervised Machine Learning: Mesothelioma
Background: Pleural Mesothelioma (PM) is an unusual, belligerent tumor that rapidly develops into cancer in the pleura of the lungs. Pleural Mesothelioma is a common type of Mesothelioma that accounts for about 75% of all Mesothelioma diagnosed yearly in the U.S. Diagnosis of Mesothelioma takes several months and is expensive. Given the risk and constraints associated with PM diagnosis, early identification of this ailment is essential for patient health. Objective: In this study, we use artificial intelligence algorithms recommending the best fit model for early diagnosis and prognosis of MPM. Methods: We retrospectively retrieved patients clinical data collected by Dicle University, Turkey, and applied multilayered perceptron (MLP), voted perceptron (VP), Clojure classifier (CC), kernel logistic regression (KLR), stochastic gradient decent SGD), adaptive boosting (AdaBoost), Hoeffding tree (VFDT), and primal estimated sub-gradient solver for support vector machine (s-Pegasos). We evaluated the models, compared and tested using paired T-test (corrected) at 0.05 significance based on their respective classification accuracy, f-measure, precision, recall, root mean squared error, receivers characteristic curve (ROC), and precision-recall curve (PRC). Results: In phase-1, SGD, AdaBoost. M1, KLR, MLP, VFDT generate optimal results with the highest possible performance measures. In phase 2, AdaBoost, with a classification accuracy of 71.29%, outperformed all other algorithms. C-reactive protein, platelet count, duration of symptoms, gender, and pleural protein were found to be the most relevant predictors that can prognosticate Mesothelioma. Conclusion: This study confirms that data obtained from Biopsy and imagining tests are strong predictors of Mesothelioma but are associated with a high cost; however, they can identify Mesothelioma with optimal accuracy.
Unsupervised Learning to Subphenotype Delirium Patients from Electronic Health Records
Delirium is a common acute onset brain dysfunction in the emergency setting and is associated with higher mortality. It is difficult to detect and monitor since its presentations and risk factors can be different depending on the underlying medical condition of patients. In our study, we aimed to identify subtypes within the delirium population and build subgroup-specific predictive models to detect delirium using Medical Information Mart for Intensive Care IV (MIMIC-IV) data. We showed that clusters exist within the delirium population. Differences in feature importance were also observed for subgroup-specific predictive models. Our work could recalibrate existing delirium prediction models for each delirium subgroup and improve the precision of delirium detection and monitoring for ICU or emergency department patients who had highly heterogeneous medical conditions.
Fast Global Convergence of Policy Optimization for Constrained MDPs
Liu, Tao, Zhou, Ruida, Kalathil, Dileep, Kumar, P. R., Tian, Chao
We address the issue of safety in reinforcement learning. We pose the problem in a discounted infinite-horizon constrained Markov decision process framework. Existing results have shown that gradient-based methods are able to achieve an $\mathcal{O}(1/\sqrt{T})$ global convergence rate both for the optimality gap and the constraint violation. We exhibit a natural policy gradient-based algorithm that has a faster convergence rate $\mathcal{O}(\log(T)/T)$ for both the optimality gap and the constraint violation. When Slater's condition is satisfied and known a priori, zero constraint violation can be further guaranteed for a sufficiently large $T$ while maintaining the same convergence rate.
Data Science 2021 : Complete Data Science & Machine Learning
Learn Complete Data Science skillset required to be a Data Scientist with all the advance concepts Master Python Programming from Basics to advance as required for Data Science and Machine Learning Learn complete Mathematics of Linear Algebra, Calculus, Vectors, Matrices for Data Science and Machine Learning. Become an expert in Statistics including Descriptive and Inferential Statistics. Become an expert in Statistics including Descriptive and Inferential Statistics. Data Science and Machine Learning are the hottest skills in demand but challenging to learn. Did you wish that there was one course for Data Science and Machine Learning that covers everything from Math for Machine Learning, Advance Statistics for Data Science, Data Processing, Machine Learning A-Z, Deep learning and more?
K-Means Clustering Algorithm
To process the learning data, the K-means algorithm in data mining starts with the first group of randomly selected centroids, which are used as the beginning points for every cluster, and then performs iterative (repetitive) calculations to optimize the positions of the centroids. You'll define a target number k, which refers to the number of centroids you need in the dataset. A centroid is the imaginary or real location representing the center of the cluster. Every data point is allocated to each of the clusters by reducing the in-cluster sum of squares. The K-means algorithm identifies k number of centroids, and then allocates every data point to the nearest cluster while keeping the centroids as small as possible.
Award details
The University of Exeter's College of Engineering, Mathematics and Physical Sciences is inviting applications for a fully-funded PhD studentship to commence in January 2022 or as soon as possible thereafter. The studentship will cover Home tuition fees plus an annual tax-free stipend of at least ยฃ15,609 for 3.5 years full-time, or pro rata for part-time study. This College studentship is open to UK and Irish nationals, who if successful in their application will receive a full studentship including payment of university tuition fees at the home fees rate. Uncertainty Quantification (UQ) for engineering models is a rapidly growing field with numerous exciting applications. However, the current best-performing algorithms for quantifying the uncertainty through Markov Chain Monte Carlo (MCMC) rely on computing a gradient that is typically not readily available for complex engineering models.
Dimensionality Reduction for Machine Learning - neptune.ai
Data forms the foundation of any machine learning algorithm, without it, Data Science can not happen. Sometimes, it can contain a huge number of features, some of which are not even required. Such redundant information makes modeling complicated. Furthermore, interpreting and understanding the data by visualization gets difficult because of the high dimensionality. This is where dimensionality reduction comes into play. Dimensionality reduction is the task of reducing the number of features in a dataset. In machine learning tasks like regression or classification, there are often too many variables to work with. These variables are also called features.
DSOR: A Scalable Statistical Filter for Removing Falling Snow from LiDAR Point Clouds in Severe Winter Weather
For autonomous vehicles to viably replace human drivers they must contend with inclement weather. Falling rain and snow introduce noise in LiDAR returns resulting in both false positive and false negative object detections. In this article we introduce the Winter Adverse Driving dataSet (WADS) collected in the snow belt region of Michigan's Upper Peninsula. WADS is the first multi-modal dataset featuring dense point-wise labeled sequential LiDAR scans collected in severe winter weather; weather that would cause an experienced driver to alter their driving behavior. We have labelled and will make available over 7 GB or 3.6 billion labelled LiDAR points out of over 26 TB of total LiDAR and camera data collected. We also present the Dynamic Statistical Outlier Removal (DSOR) filter, a statistical PCL-based filter capable or removing snow with a higher recall than the state of the art snow de-noising filter while being 28\% faster. Further, the DSOR filter is shown to have a lower time complexity compared to the state of the art resulting in an improved scalability. Our labeled dataset and DSOR filter will be made available at https://bitbucket.org/autonomymtu/dsor_filter