Performance Analysis
Confusion Matrix and it's 25 offspring: or the link between machine learning and epidemiology Dr. Yury Zablotski
For instance, an LR of 3 suggests that for every false positive, there are 3 true positives. The greater the value of the LR for a particular test, the more likely a positive test result is a true positive. On the other hand, an LR 1 would imply that an individual with a positive test result is more likely to be non-diseased than diseased. The rationale for the diagnostic odds ratio is that it is a single indicator of test performance (like accuracy and Youden's J index, explained below) which is independent of prevalence (unlike accuracy) and is presented as an odds ratio, which is familiar to epidemiologists. Similarly to a usual odds ratio, the diagnostic odds ratio ranges from zero to infinity, where DOR greater then one is already good, and the higher DOR goes, the better the test performs. DOR of less than one indicates that the test performs bad, or even gives wrong information.
Machine Learning for Exploring Spatial Affordance Patterns
This dissertation uses supervised and unsupervised data mining techniques to analyse office floor plans in an attempt to gain a better understanding of their geometry-to-function relationship. This question was deemed relevant after a background review of the state-of-the-art in automated floor-plan generation tools showed that such tools have been prototyped since the 1960s, but their search space is ill-informed because there are few formalisms to describe spatial affordance. To show and evaluate the relationship of geometry and use, data from visual graph analysis were used to train three supervised learners and compare these to a baseline accuracy established with a ZeroR classifier. This showed that for the office dataset examined, visual mean depth and integration are most tightly linked to usage and that the supervised learning algorithm J48 can correctly predict class performance on unseen examples to up to 79.5%. The thesis also includes an evaluation of the layout case studies with unsupervised learners, which showed that use could not be immediately reverse-engineered based solemnly on the VGA information to achieve a strong cluster-to-class evaluation.
Improving Classification Accuracy by Mining Deterministic and Frequent Rules
Huang, Yuxiao (George Washington University )
Patterns underlying the data sometimes take the form of IF conditions THEN outcome. However, not all the classifiers can detect such rules, resulting in compromised classification accuracy. In this paper we proposed an Add-on Rule-based Classifier (ARC) that can be paired with any existing classifier (base). The idea of ARC is improving the accuracy of the base by 1) mining deterministic and frequent rules, and 2) using such rules to assist the base in classification. Key novelty includes 1) a greedy search algorithm that identifies the rules by alternating between adding the ``best'' condition and removing the ``worst", and 2) new heuristics for selecting the best and worst conditions. We theoretically proved that rules detected by ARC are sound, complete, and minimal, indicating that ARC will almost never degrade the accuracy of the base, but instead, could often improve it. To experimentally verify this claim, we paired ARC with 9 leading classifiers and tested the ensembles on 12 UCI datasets. Empirical results show that, ARC never lowers the accuracy of the base and, more importantly, usually increases it (where some of the increases are statistically significant), echoing what we theoretically proved.
FDA investigates COVID-19 test with false negatives
Food and Drug Administration Commissioner Steve Hahn says it will be up to the White House to determine whether it continues to use a coronavirus test that has falsely cleared patients of infection. Hahn Told CBS on Friday the FDA will keep "providing guidance to the White House regarding this test" but whether to keep using the test "will be a White House decision." The test is used daily at the White House to test President Donald Trump and key members of his staff, including the coronavirus task force. The FDA said late Thursday it was investigating preliminary data suggesting Abbott Laboratories' 15-minute test can miss COVID-19 cases, producing false negatives. Hahn told CBS the test is on the market and the FDA continues to "recommend its use or to have it available for use."
The Latest: FDA Investigating Test With False Negatives
Under head of state and ruling Communist Party leader Xi Jinping's leadership, China has been able to "put the outbreak under control through arduous efforts and has been gradually resuming economic and social life while undertaking prevention and control measures on a regular basis," Wang was quoted as saying in a phone call Thursday with the foreign ministers of Hungary, Estonia and Bosnia and Herzegovina.
Adaptive XGBoost for Evolving Data Streams
Montiel, Jacob, Mitchell, Rory, Frank, Eibe, Pfahringer, Bernhard, Abdessalem, Talel, Bifet, Albert
Boosting is an ensemble method that combines base models in a sequential manner to achieve high predictive accuracy. A popular learning algorithm based on this ensemble method is eXtreme Gradient Boosting (XGB). We present an adaptation of XGB for classification of evolving data streams. In this setting, new data arrives over time and the relationship between the class and the features may change in the process, thus exhibiting concept drift. The proposed method creates new members of the ensemble from mini-batches of data as new data becomes available. The maximum ensemble size is fixed, but learning does not stop when this size is reached because the ensemble is updated on new data to ensure consistency with the current concept. We also explore the use of concept drift detection to trigger a mechanism to update the ensemble. We test our method on real and synthetic data with concept drift and compare it against batch-incremental and instance-incremental classification methods for data streams.
Automatic Dialogic Instruction Detection for K-12 Online One-on-one Classes
Xu, Shiting, Ding, Wenbiao, Liu, Zitao
Online one-on-one class is created for highly interactive and immersive learning experience. It demands a large number of qualified online instructors. In this work, we develop six dialogic instructions and help teachers achieve the benefits of one-on-one learning paradigm. Moreover, we utilize neural language models, i.e., long short-term memory (LSTM), to detect above six instructions automatically. Experiments demonstrate that the LSTM approach achieves AUC scores from 0.840 to 0.979 among all six types of instructions on our real-world educational dataset.
Statistical Equity: A Fairness Classification Objective
Mehrabi, Ninareh, Huang, Yuzhong, Morstatter, Fred
Machine learning systems have been shown to propagate the societal errors of the past. In light of this, a wealth of research focuses on designing solutions that are "fair." Even with this abundance of work, there is no singular definition of fairness, mainly because fairness is subjective and context dependent. We propose a new fairness definition, motivated by the principle of equity, that considers existing biases in the data and attempts to make equitable decisions that account for these previous historical biases. We formalize our definition of fairness, and motivate it with its appropriate contexts. Next, we operationalize it for equitable classification. We perform multiple automatic and human evaluations to show the effectiveness of our definition and demonstrate its utility for aspects of fairness, such as the feedback loop.
Simultaneous imputation and disease classification in incomplete medical datasets using Multigraph Geometric Matrix Completion (MGMC)
Vivar, Gerome, Kazi, Anees, Burwinkel, Hendrik, Zwergal, Andreas, Navab, Nassir, Ahmadi, Seyed-Ahmad
Large-scale population-based studies in medicine are a key resource towards better diagnosis, monitoring, and treatment of diseases. They also serve as enablers of clinical decision support systems, in particular Computer Aided Diagnosis (CADx) using machine learning (ML). Numerous ML approaches for CADx have been proposed in literature. However, these approaches assume full data availability, which is not always feasible in clinical data. To account for missing data, incomplete data samples are either removed or imputed, which could lead to data bias and may negatively affect classification performance. As a solution, we propose an end-to-end learning of imputation and disease prediction of incomplete medical datasets via Multigraph Geometric Matrix Completion (MGMC). MGMC uses multiple recurrent graph convolutional networks, where each graph represents an independent population model based on a key clinical meta-feature like age, sex, or cognitive function. Graph signal aggregation from local patient neighborhoods, combined with multigraph signal fusion via self-attention, has a regularizing effect on both matrix reconstruction and classification performance. Our proposed approach is able to impute class relevant features as well as perform accurate classification on two publicly available medical datasets. We empirically show the superiority of our proposed approach in terms of classification and imputation performance when compared with state-of-the-art approaches. MGMC enables disease prediction in multimodal and incomplete medical datasets. These findings could serve as baseline for future CADx approaches which utilize incomplete datasets.
AI and Machine Learning in Cyber Security
Zen monks have been using a tool called a'koan' for hundreds of years to assist them in reaching enlightenment. These koans are like riddles or stories that can only be solved by letting go of ones narrowing believes and stories about how things should be. Zen students sit in silent meditation and observe how the koan is working on them, slowly transforming their way of looking at the world and revealing a tiny piece of the path to nirvana, that place of no suffering. "Zen is like a man hanging by his teeth in a tree over a precipice. His hands grasp no branch, his feet rest on no limb, and under the tree another man asks him, 'Why did Bodhidharma come to China from the West?' If the man in the tree does not answer, he misses the question, and if he answers, he falls and loses his life. Now what shall he do?" -- Zen Koan -- Case 5 of the Gateless Gate Collection.