Performance Analysis
DeepPass -- Finding Passwords With Deep Learning
This is definitely more manageable, with 723 false positives out of 400,000 balanced input words. False positives and false negatives are intrinsically linked, and reducing one will increase the other. Using class weighting to reduce false positives here means that false negatives, or potential passwords we miss, will increase! However I felt that this tradeoff was necessary for the approach to hope to be usable.
Automate ML Development With Amazon Sagemaker - Analytics Vidhya
This article was published as a part of the Data Science Blogathon. Amazon Sagemaker is arguably the most powerful, feature-rich, and fully managed machine learning service developed by Amazon. From creating your own labeled datasets to deploying and monitoring the models on production, Sagemaker is equipped to do everything. It can also provide an integrated Jupyter notebook instance for easy access to your data for exploration and analysis, so you don't have to fiddle around with server configuration. Sagemaker supports bring-your-own-algorithms and frameworks, which offer flexible distributed training options that adjust to your specific workflows.
Predicting Political Ideology from Digital Footprints
Kitchener, Michael, Anantharama, Nandini, Angus, Simon D., Raschky, Paul A.
This paper proposes a new method to predict individual political ideology from digital footprints on one of the world's largest online discussion forum. We compiled a unique data set from the online discussion forum reddit that contains information on the political ideology of around 91,000 users as well as records of their comment frequency and the comments' text corpus in over 190,000 different subforums of interest. Applying a set of statistical learning approaches, we show that information about activity in non-political discussion forums alone, can very accurately predict a user's political ideology. Depending on the model, we are able to predict the economic dimension of ideology with an accuracy of up to 90.63% and the social dimension with and accuracy of up to 82.02%. In comparison, using the textual features from actual comments does not improve predictive accuracy. Our paper highlights the importance of revealed digital behaviour to complement stated preferences from digital communication when analysing human preferences and behaviour using online data.
Learning brain MRI quality control: a multi-factorial generalization problem
Reguig, Ghiles, Chupin, Marie, Dary, Hugo, Bardinet, Eric, Lehรฉricy, Stรฉphane, Valabregue, Romain
Due to the growing number of MRI data, automated quality control (QC) has become essential, especially for larger scale analysis. Several attempts have been made in order to develop reliable and scalable QC pipelines. However, the generalization of these methods on new data independent of those used for learning is a difficult problem because of the biases inherent in MRI data. This work aimed at evaluating the performances of the MRIQC pipeline on various large-scale datasets (ABIDE, N = 1102 and CATI derived datasets, N = 9037) used for both training and evaluation purposes. We focused our analysis on the MRIQC preprocessing steps and tested the pipeline with and without them. We further analyzed the site-wise and study-wise predicted classification probability distributions of the models without preprocessing trained on ABIDE and CATI data. Our main results were that a model using features extracted from MRIQC without preprocessing yielded the best results when trained and evaluated on large multi-center datasets with a heterogeneous population (an improvement of the ROC-AUC score on unseen data of 0.10 for the model trained on a subset of the CATI dataset). We concluded that a model trained with data from a heterogeneous population, such as the CATI dataset, provides the best scores on unseen data. In spite of the performance improvement, the generalization abilities of the models remain questionable when looking at the site-wise/study-wise probability predictions and the optimal classification threshold derived from them.
How much does a bootstrap estimate depend on the random number stream?
Many modern statistical techniques incorporate randomness: simulation, bootstrapping, random forests, and so forth. To use the technique, you need to specify a seed value, which determines pseudorandom numbers that are used in the algorithm. Consequently, the seed value also determines the results of the algorithm. In theory, if you know the seed value and the internal details of the pseudorandom algorithm, then the stream is completely determined, and the results of an algorithm are reproducible. For example, if I publish code for a simulation or bootstrap method in SAS, you can reproduce my computations as long as my program specifies the seed value for every part of the program that uses random numbers.
OSegNet: Operational Segmentation Network for COVID-19 Detection using Chest X-ray Images
Degerli, Aysen, Kiranyaz, Serkan, Chowdhury, Muhammad E. H., Gabbouj, Moncef
Coronavirus disease 2019 (COVID-19) has been diagnosed automatically using Machine Learning algorithms over chest X-ray (CXR) images. However, most of the earlier studies used Deep Learning models over scarce datasets bearing the risk of overfitting. Additionally, previous studies have revealed the fact that deep networks are not reliable for classification since their decisions may originate from irrelevant areas on the CXRs. Therefore, in this study, we propose Operational Segmentation Network (OSegNet) that performs detection by segmenting COVID-19 pneumonia for a reliable diagnosis. To address the data scarcity encountered in training and especially in evaluation, this study extends the largest COVID-19 CXR dataset: QaTa-COV19 with 121,378 CXRs including 9258 COVID-19 samples with their corresponding ground-truth segmentation masks that are publicly shared with the research community. Consequently, OSegNet has achieved a detection performance with the highest accuracy of 99.65% among the state-of-the-art deep models with 98.09% precision.
How to Test a Recommender System - neptune.ai
Recommender systems fundamentally address the question โ What do people want? Although it is an extensive question, in the context of a consumer application like e-commerce, the answer could be to serve the best products in terms of price and quality for a consumer. For a news aggregator website, it could be to show reliable and relevant content. In a case where a user would have to look through thousands or millions of items to find what they are looking for, a recommendation engine is indispensable. The engine filters over 3,000 titles at a time using 1,300 recommendation clusters based on user preferences. It is so accurate that personalised recommendations from the engine drive 80% of Netflix viewer activity. However, building and evaluating a recommender system is very different compared to a single ML model regarding design decisions, engineering, and metrics. In this article, we will focus on testing a recommendation system. The second and third require a lot of user-item interaction data. If that is not available, one might start with the first type of recommender system.
Bird Species Classification with Machine Learning
Here I manually saved the column names, which are numerical and categorical, and also saved the target column. From the info function, there seem to be missing values, and we can see that location and sex should be categorical, so we have to do some data type conversion later on. Let's first visualize our target class. We see location and species seemingly for their respective locations and species (loc2 & species C, loc3 & species A). We also see there are slightly more female (1) birds than the male counterpart.
Toward Reduction in False-Positive Thyroid Nodule Biopsies with a Deep Learningโbased Risk Stratification System Using US Cine-Clip Images
The Cine-CNNTrans achieved an average AUC of 0.88 0.10 for classifying benign versus malignant thyroid nodules. The Cine-CNNTrans showed higher AUC than the Static-2DCNN (P .03). For aggregating framewise outputs into nodulewise scores, the Cine-CNNTrans tended toward higher AUC compared with the Cine-CNNAvePool (P .17). Our system tended toward higher AUC than the Cine-Radiomics and the ACR TI-RADS level, though the difference did not achieve statistical significance (P .16
Critical appraisal of artificial intelligence-based prediction models for cardiovascular disease
The medical field has seen a rapid increase in the development of artificial intelligence (AI)-based prediction models. With the introduction of such AI-based prediction model tools and software in cardiovascular patient care, the cardiovascular researcher and healthcare professional are challenged to understand the opportunities as well as the limitations of the AI-based predictions. In this article, we present 12 critical questions for cardiovascular health professionals to ask when confronted with an AI-based prediction model. We aim to support medical professionals to distinguish the AI-based prediction models that can add value to patient care from the AI that does not. Listen to the audio abstract of this contribution. Artificial intelligence (AI) and its subdiscipline machine learning are receiving increasing attention throughout medicine, including cardiovascular medicine.1,2 Proponents promise AI will change the way medicine and healthcare is practiced, by making use of technological advancements that allow for collection of increasingly detailed and diverse data and the ever-increasing computational ability to analyse and combine such data. An important part of these promises is the development and implementation of more accurate clinical prediction models (algorithms, tools, or rules, from here onwards simply referred to as prediction models) to improve--or according to some advocates, even revolutionize--screening, diagnosis, and prognostication of diseases.