Goto

Collaborating Authors

 Oceania


Spectral Methods for Data Science: A Statistical Perspective

arXiv.org Machine Learning

Spectral methods have emerged as a simple yet surprisingly effective approach for extracting information from massive, noisy and incomplete data. In a nutshell, spectral methods refer to a collection of algorithms built upon the eigenvalues (resp. singular values) and eigenvectors (resp. singular vectors) of some properly designed matrices constructed from data. A diverse array of applications have been found in machine learning, data science, and signal processing. Due to their simplicity and effectiveness, spectral methods are not only used as a stand-alone estimator, but also frequently employed to initialize other more sophisticated algorithms to improve performance. While the studies of spectral methods can be traced back to classical matrix perturbation theory and methods of moments, the past decade has witnessed tremendous theoretical advances in demystifying their efficacy through the lens of statistical modeling, with the aid of non-asymptotic random matrix theory. This monograph aims to present a systematic, comprehensive, yet accessible introduction to spectral methods from a modern statistical perspective, highlighting their algorithmic implications in diverse large-scale applications. In particular, our exposition gravitates around several central questions that span various applications: how to characterize the sample efficiency of spectral methods in reaching a target level of statistical accuracy, and how to assess their stability in the face of random noise, missing data, and adversarial corruptions? In addition to conventional $\ell_2$ perturbation analysis, we present a systematic $\ell_{\infty}$ and $\ell_{2,\infty}$ perturbation theory for eigenspace and singular subspaces, which has only recently become available owing to a powerful "leave-one-out" analysis framework.


Detection of Anomalies in a Time Series Data using InfluxDB and Python

arXiv.org Machine Learning

Analysis of water and environmental data is an important aspect of many intelligent water and environmental system applications where inference from such analysis plays a significant role in decision making. Quite often these data that are collected through sensible sensors can be anomalous due to different reasons such as systems breakdown, malfunctioning of sensor detectors, and more. Regardless of their root causes, such data severely affect the results of the subsequent analysis. This paper demonstrates data cleaning and preparation for time-series data and further proposes cost-sensitive machine learning algorithms as a solution to detect anomalous data points in time-series data. The following models: Logistic Regression, Random Forest, Support Vector Machines have been modified to support the cost-sensitive learning which penalizes misclassified samples thereby minimizing the total misclassification cost. Our results showed that Random Forest outperformed the rest of the models at predicting the positive class (i.e anomalies). Applying predictive model improvement techniques like data oversampling seems to provide little or no improvement to the Random Forest model. Interestingly, with recursive feature elimination, we achieved a better model performance thereby reducing the dimensions in the data. Finally, with Influxdb and Kapacitor the data was ingested and streamed to generate new data points to further evaluate the model performance on unseen data, this will allow for early recognition of undesirable changes in the drinking water quality and will enable the water supply companies to rectify on a timely basis whatever undesirable changes abound.


AI-powered construction software company to create 50 Irish jobs Technology, news for Ireland, Employment,Ireland,Technology,

#artificialintelligence

DBIC Ventures, the venture arm of Dublin BIC, today announces details of a co-investment in AI-powered construction software company Evercam, representing the first investment by DBIC Ventures' new seed and early stage fund which is backed by Enterprise Ireland and a number of leading Irish technology entrepreneurs and business leaders. The investment round totalled โ‚ฌ600,000 and will support the creation of 50 new jobs and the company's continued strong growth in international markets. It is the first investment by DBIC Ventures' latest fund which plans to back around 30 leading high growth early stage Irish tech companies over the next four years. Elkstone, the Irish multi-family office, is a co-investor in the round. Evercam enhances construction site productivity by improving project visibility with verifiable intelligence from high-resolution time-lapse cameras located on client sites.


Intel Labs Moving Mountains With Neuromorphic Computing And Photonics Technologies

#artificialintelligence

While the industry loves to combine "R&D" and we see this in every tech company's P&L, research and development are very different. Research is high risk, market making investments and discoveries that are unattached to products. Development is applying that research and other's IP to create an end product or services. Very few companies do research, and Intel has had a heritage in research for decades. One of the most exciting aspects of working as a tech analyst is, quite frankly, being one of the first to learn of these new, research-driven, cutting-edge technologies coming down the pipeline in the not-so-distant future--from the expected to the truly mind-boggling.


How AI and data science can help fight COVID-19 - The Data Scientist

#artificialintelligence

The world is currently facing an unprecedented health crisis. The coronavirus known as COVID-19 has already taken the lives of thousands of people worldwide and has infected many more. At the time of writing, the crisis is not over yet, and there are concerns that it might lead into a new financial crisis. At these times of crisis, humanity needs any weapon at its disposal. So, what can AI and data science do in order to help with this crisis?


Fish flock to artificial 'Reefpyramids' off Darwin

#artificialintelligence

Bigger boats, better equipment, accurate weather forecasts, social media and climate change are threatening fish populations. Four artificial reefs have been sitting off the NT coast for a year, and early results are promising for the $8.3 million project. But environmentalists wish the NT Government would invest "as much energy, money and enthusiasm into existing reefs", which have shown evidence of coral bleaching. A total of 116 Reefpyramids, weighing 24 tonnes each, have been dropped across four sites. NT Fisheries aquatic biosecurity liaison Evan Needham said the reefs were positioned on barren seabed around Darwin to provide fish-breeding areas and destinations for fishers.


Combining Visual and Textual Features for Semantic Segmentation of Historical Newspapers

arXiv.org Artificial Intelligence

The massive amounts of digitized historical documents acquired over the last decades naturally lend themselves to automatic processing and exploration. Research work seeking to automatically process facsimiles and extract information thereby are multiplying with, as a first essential step, document layout analysis. If the identification and categorization of segments of interest in document images have seen significant progress over the last years thanks to deep learning techniques, many challenges remain with, among others, the use of finer-grained segmentation typologies and the consideration of complex, heterogeneous documents such as historical newspapers. Besides, most approaches consider visual features only, ignoring textual signal. In this context, we introduce a multimodal approach for the semantic segmentation of historical newspapers that combines visual and textual features. Based on a series of experiments on diachronic Swiss and Luxembourgish newspapers, we investigate, among others, the predictive power of visual and textual features and their capacity to generalize across time and sources. Results show consistent improvement of multimodal models in comparison to a strong visual baseline, as well as better robustness to high material variance.


COVID-19 Image Data Collection: Prospective Predictions Are the Future

arXiv.org Artificial Intelligence

Across the world's coronavirus disease 2019 (COVID-19) hot spots, the need to streamline patient diagnosis and management has become more pressing than ever. As one of the main imaging tools, chest X-rays (CXRs) are common, fast, non-invasive, relatively cheap, and potentially bedside to monitor the progression of the disease. This paper describes the first public COVID-19 image data collection as well as a preliminary exploration of possible use cases for the data. This dataset currently contains hundreds of frontal view X-rays and is the largest public resource for COVID-19 image and prognostic data, making it a necessary resource to develop and evaluate tools to aid in the treatment of COVID-19. It was manually aggregated from publication figures as well as various web based repositories into a machine learning (ML) friendly format with accompanying dataloader code. We collected frontal and lateral view imagery and metadata such as the time since first symptoms, intensive care unit (ICU) status, survival status, intubation status, or hospital location.


Towards Accurate Spatiotemporal COVID-19 Risk Scores using High Resolution Real-World Mobility Data

arXiv.org Machine Learning

As countries look towards re-opening of economic activities amidst the ongoing COVID-19 pandemic, ensuring public health has been challenging. While contact tracing only aims to track past activities of infected users, one path to safe reopening is to develop reliable spatiotemporal risk scores to indicate the propensity of the disease. Existing works which aim to develop risk scores either rely on compartmental model-based reproduction numbers (which assume uniform population mixing) or develop coarse-grain spatial scores based on reproduction number (R0) and macro-level density-based mobility statistics. Instead, in this paper, we develop a Hawkes process-based technique to assign relatively fine-grain spatial and temporal risk scores by leveraging high-resolution mobility data based on cell-phone originated location signals. While COVID-19 risk scores also depend on a number of factors specific to an individual, including demography and existing medical conditions, the primary mode of disease transmission is via physical proximity and contact. Therefore, we focus on developing risk scores based on location density and mobility behaviour. We demonstrate the efficacy of the developed risk scores via simulation based on real-world mobility data. Our results show that fine-grain spatiotemporal risk scores based on high-resolution mobility data can provide useful insights and facilitate safe re-opening.


Exponential Lower Bounds for Batch Reinforcement Learning: Batch RL can be Exponentially Harder than Online RL

arXiv.org Artificial Intelligence

Several practical applications of reinforcement learning involve an agent learning from past data without the possibility of further exploration. Often these applications require us to 1) identify a near optimal policy or to 2) estimate the value of a target policy. For both tasks we derive exponential information-theoretic lower bounds in discounted infinite horizon MDPs with a linear function representation for the action value function even if 1) realizability holds, 2) the batch algorithm observes the exact reward and transition functions, and 3) the batch algorithm is given the best a priori data distribution for the problem class. Furthermore, if the dataset does not come from policy rollouts then the lower bounds hold even if all policies admit a linear representation. If the objective is to find a near-optimal policy, we discover that these hard instances are easily solved by an online algorithm, showing that there exist RL problems where batch RL is exponentially harder than online RL even under the most favorable batch data distribution. In other words, online exploration is critical to enable sample efficient RL with function approximation. A second corollary is the exponential separation between finite and infinite horizon batch problems under our assumptions. On a technical level, this work helps formalize the issue known as deadly triad and explains that the bootstrapping problem is potentially more severe than the extrapolation issue for RL because unlike the latter, bootstrapping cannot be mitigated by adding more samples.