Goto

Collaborating Authors

 Accuracy


Predicting traffic overflows on private peering

arXiv.org Artificial Intelligence

Large content providers and content distribution network operators usually connect with large Internet service providers (eyeball networks) through dedicated private peering. The capacity of these private network interconnects is provisioned to match the volume of the real content demand by the users. Unfortunately, in case of a surge in traffic demand, for example due to a content trending in a certain country, the capacity of the private interconnect may deplete and the content provider/distributor would have to reroute the excess traffic through transit providers. Although, such overflow events are rare, they have significant negative impacts on content providers, Internet service providers, and end-users. These include unexpected delays and disruptions reducing the user experience quality, as well as direct costs paid by the Internet service provider to the transit providers. If the traffic overflow events could be predicted, the Internet service providers would be able to influence the routes chosen for the excess traffic to reduce the costs and increase user experience quality. In this article we propose a method based on an ensemble of deep learning models to predict overflow events over a short term horizon of 2-6 hours and predict the specific interconnections that will ingress the overflow traffic. The method was evaluated with 2.5 years' traffic measurement data from a large European Internet service provider resulting in a true-positive rate of 0.8 while maintaining a 0.05 false-positive rate. The lockdown imposed by the COVID-19 pandemic reduced the overflow prediction accuracy. Nevertheless, starting from the end of April 2020 with the gradual lockdown release, the old models trained before the pandemic perform equally well.


Train, evaluate, monitor, infer: End-to-end machine learning in Elastic

#artificialintelligence

Machine learning pipelines have evolved tremendously in the past several years. With a wide variety of tools and frameworks out there to simplify building, training, and deployment, the turnaround time on machine learning model development has improved drastically. However, even with all these simplifications, there is still a steep learning curve associated with a lot of these tools. In order to use machine learning in the Elastic Stack, all you really need is for your data to be stored in Elasticsearch. Once there, extracting valuable insights from your data is as simple as clicking a few buttons in Kibana.


Airlines look to help ailing industry with coronavirus testing at airports (but it's not a cure-all)

Los Angeles Times

Financially strapped airlines are pushing an idea intended to breathe new life into the travel industry: coronavirus tests that passengers can take before boarding a flight. Several airlines, including United, American, Hawaiian, JetBlue and Alaska, have announced plans to begin offering testing -- either kits mailed to a passenger's home or rapid tests taken at or near airports -- that would allow travelers to enter specific states and countries without having to quarantine. The tests will cost fliers $90 to $250, depending on the airline and the type of test. At Los Angeles International Airport, a design company has announced plans to convert cargo containers into a coronavirus testing facility with an on-site lab that can produce results in about two hours. On Thursday, Tampa International Airport began offering testing to all arriving and departing passengers on a walk-in basis. It's an idea that has gone global, with a trade group for the world's airlines calling on governments to create a testing standard for airline passengers as a way to fight the COVID-19 pandemic instead of using travel restrictions and mandatory quarantines.


Neural Bootstrapper

arXiv.org Machine Learning

Bootstrapping has been a primary tool for uncertainty quantification, and their theoretical and computational properties have been investigated in the field of statistics and machine learning. However, due to its nature of repetitive computations, the computational burden required to implement bootstrap procedures for the neural network is painfully heavy, and this fact seriously hurdles the practical use of these procedures on the uncertainty estimation of modern deep learning. To overcome the inconvenience, we propose a procedure called Neural Bootstrapper (NeuBoots). We reveal that the NeuBoots stably generate valid bootstrap samples that coincide with the desired target samples with minimal extra computational cost compared to traditional bootstrapping. Consequently, NeuBoots makes it feasible to construct bootstrap confidence intervals of outputs of neural networks and quantify their predictive uncertainty. We also suggest NeuBoots for deep convolutional neural networks to consider its utility in image classification tasks, including calibration, detection of out-of-distribution samples, and active learning. Empirical results demonstrate that NeuBoots is significantly beneficial for the above purposes. Since the introduction of the nonparametric bootstrap (Efron, 1979), bootstrap (or bagging) procedures have been commonly used as a primary tool in quantifying uncertainty lying on statistical inference, e.g.


Linear Classifier Combination via Multiple Potential Functions

arXiv.org Machine Learning

A vital aspect of the classification based model construction process is the calibration of the scoring function. One of the weaknesses of the calibration process is that it does not take into account the information about the relative positions of the recognized objects in the feature space. To alleviate this limitation, in this paper, we propose a novel concept of calculating a scoring function based on the distance of the object from the decision boundary and its distance to the class centroid. An important property is that the proposed score function has the same nature for all linear base classifiers, which means that outputs of these classifiers are equally represented and have the same meaning. The proposed approach is compared with other ensemble algorithms and experiments on multiple Keel datasets demonstrate the effectiveness of our method. To discuss the results of our experiments, we use multiple classification performance measures and statistical analysis.


Quasar Detection using Linear Support Vector Machine with Learning From Mistakes Methodology

arXiv.org Machine Learning

The field of Astronomy requires the collection and assimilation of vast volumes of data. The data handling and processing problem has become severe as the sheer volume of data produced by scientific instruments each night grows exponentially. This problem becomes extensive for conventional methods of processing the data, which was mostly manual, but is the perfect setting for the use of Machine Learning approaches. While building classifiers for Astronomy, the cost of losing a rare object like supernovae or quasars to detection losses is far more severe than having many false positives, given the rarity and scientific value of these objects. In this paper, a Linear Support Vector Machine (LSVM) is explored to detect Quasars, which are extremely bright objects in which a supermassive black hole is surrounded by a luminous accretion disk. In Astronomy, it is vital to correctly identify quasars, as they are very rare in nature. Their rarity creates a class-imbalance problem that needs to be taken into consideration. The class-imbalance problem and high cost of misclassification are taken into account while designing the classifier. To achieve this detection, a novel classifier is explored, and its performance is evaluated. It was observed that LSVM along with Ensemble Bagged Trees (EBT) achieved a 10x reduction in the False Negative Rate, using the Learning from Mistakes methodology.


Logistic Regression Clearly Explained

#artificialintelligence

Logistic Regression is the most widely used classification algorithm in machine learning. It is used in many real-world scenarios like spam detected, cancer detection, IRIS dataset, etc. Mostly it is used in binary classification problems. But it can also be used in multiclass classification. Logistic Regression predicts the probability that the given data point belongs to a certain class or not. In this article, I will be using the famous heart disease dataset from Kaggle. In this dataset, the main goal is to predict whether the given person has heart disease or not.


A survey on natural language processing (nlp) and applications in insurance

arXiv.org Machine Learning

Text is the most widely used means of communication today. This data is abundant but nevertheless complex to exploit within algorithms. For years, scientists have been trying to implement different techniques that enable computers to replicate some mechanisms of human reading. During the past five years, research disrupted the capacity of the algorithms to unleash the value of text data. It brings today, many opportunities for the insurance industry.Understanding those methods and, above all, knowing how to apply them is a major challenge and key to unleash the value of text data that have been stored for many years. Processing language with computer brings many new opportunities especially in the insurance sector where reports are central in the information used by insurers. SCOR's Data Analytics team has been working on the implementation of innovative tools or products that enable the use of the latest research on text analysis. Understanding text mining techniques in insurance enhances the monitoring of the underwritten risks and many processes that finally benefit policyholders.This article proposes to explain opportunities that Natural Language Processing (NLP) are providing to insurance. It details different methods used today in practice traces back the story of them. We also illustrate the implementation of certain methods using open source libraries and python codes that we have developed to facilitate the use of these techniques.After giving a general overview on the evolution of text mining during the past few years,we share about how to conduct a full study with text mining and share some examples to serve those models into insurance products or services. Finally, we explained in more details every step that composes a Natural Language Processing study to ensure the reader can have a deep understanding on the implementation.


When will the mist clear? On the Interpretability of Machine Learning for Medical Applications: a survey

arXiv.org Artificial Intelligence

Artificial Intelligence is providing astonishing results, with medicine being one of its favourite playgrounds. In a few decades, computers may be capable of formulating diagnoses and choosing the correct treatment, while robots may perform surgical operations, and conversational agents could interact with patients as virtual coaches. Machine Learning and, in particular, Deep Neural Networks are behind this revolution. In this scenario, important decisions will be controlled by standalone machines that have learned predictive models from provided data. Among the most challenging targets of interest in medicine are cancer diagnosis and therapies but, to start this revolution, software tools need to be adapted to cover the new requirements. In this sense, learning tools are becoming a commodity in Python and Matlab libraries, just to name two, but to exploit all their possibilities, it is essential to fully understand how models are interpreted and which models are more interpretable than others. In this survey, we analyse current machine learning models, frameworks, databases and other related tools as applied to medicine - specifically, to cancer research - and we discuss their interpretability, performance and the necessary input data. From the evidence available, ANN, LR and SVM have been observed to be the preferred models. Besides, CNNs, supported by the rapid development of GPUs and tensor-oriented programming libraries, are gaining in importance. However, the interpretability of results by doctors is rarely considered which is a factor that needs to be improved. We therefore consider this study to be a timely contribution to the issue.


MARS-Gym: A Gym framework to model, train, and evaluate Recommender Systems for Marketplaces

arXiv.org Machine Learning

Recommender Systems are especially challenging for marketplaces since they must maximize user satisfaction while maintaining the healthiness and fairness of such ecosystems. In this context, we observed a lack of resources to design, train, and evaluate agents that learn by interacting within these environments. For this matter, we propose MARS-Gym, an open-source framework to empower researchers and engineers to quickly build and evaluate Reinforcement Learning agents for recommendations in marketplaces. MARS-Gym addresses the whole development pipeline: data processing, model design and optimization, and multi-sided evaluation. We also provide the implementation of a diverse set of baseline agents, with a metrics-driven analysis of them in the Trivago marketplace dataset, to illustrate how to conduct a holistic assessment using the available metrics of recommendation, off-policy estimation, and fairness. With MARS-Gym, we expect to bridge the gap between academic research and production systems, as well as to facilitate the design of new algorithms and applications.