Performance Analysis
A quantitative assessment of the effect of different algorithmic schemes to the task of learning the structure of Bayesian Networks
Beretta, Stefano, Castelli, Mauro, Goncalves, Ivo, Ramazzotti, Daniele
One of the most challenging tasks when adopting Bayesian Networks (BNs) is the one of learning their structure from data. This task is complicated by the huge search space of possible solutions and turned out to be a well-known NP-hard problem and, hence, approximations are required. However, to the best of our knowledge, a quantitative analysis of the performance and characteristics of the different heuristics to solve this problem has never been done before. For this reason, in this work, we provide a detailed study of the different state-of-the-arts methods for structural learning on simulated data considering both BNs with discrete and continuous variables, and with different rates of noise in the data. In particular, we investigate the characteristics of different widespread scores proposed for the inference and the statistical pitfalls within them.
Prediction of Daytime Hypoglycemic Events Using Continuous Glucose Monitoring Data and Classification Technique
Jung, Miyeon, Lee, You-Bin, Jin, Sang-Man, Park, Sung-Min
Daytime hypoglycemia should be accurately predicted to achieve normoglycemia and to avoid disastrous situations. Hypoglycemia, an abnormally low blood glucose level, is divided into daytime hypoglycemia and nocturnal hypoglycemia. Many studies of hypoglycemia prevention deal with nocturnal hypoglycemia. In this paper, we propose new predictor variables to predict daytime hypoglycemia using continuous glucose monitoring (CGM) data. We apply classification and regression tree (CART) as a prediction method. The independent variables of our prediction model are the rate of decrease from a peak and absolute level of the BG at the decision point. The evaluation results showed that our model was able to detect almost 80% of hypoglycemic events 15 min in advance, which was higher than the existing methods with similar conditions. The proposed method might achieve a real-time prediction as well as can be embedded into BG monitoring device.
Optimal client recommendation for market makers in illiquid financial products
Hendricks, Dieter, Roberts, Stephen J.
The process of liquidity provision in financial markets can result in prolonged exposure to illiquid instruments for market makers. In this case, where a proprietary position is not desired, pro-actively targeting the right client who is likely to be interested can be an effective means to offset this position, rather than relying on commensurate interest arising through natural demand. In this paper, we consider the inference of a client profile for the purpose of corporate bond recommendation, based on typical recorded information available to the market maker. Given a historical record of corporate bond transactions and bond meta-data, we use a topic-modelling analogy to develop a probabilistic technique for compiling a curated list of client recommendations for a particular bond that needs to be traded, ranked by probability of interest. We show that a model based on Latent Dirichlet Allocation offers promising performance to deliver relevant recommendations for sales traders.
100 Data Science Interview Questions and Answers (General) for 2017
In collaboration with data scientists, industry experts and top counsellors, we have put together a list of general data science interview questions and answers to help you with your preparation in applying for data science jobs. This also includes a list of open ended questions that interviewers ask to get a feel of how often and how quickly you can think on your feet.There are some data analyst interview questions in this blog which can also be asked in a data science interview. These kind of analytics interview questions also measure if you were successful in applying data science techniques to real life problems. If you would like more information about Online Data Science course, please click the orange "Request Info" button on top of this page. Data Science is not an easy field to get into. This is something all data scientists will agree on. Apart from having a degree in mathematics/statistics or engineering, a data scientist also needs to go through intense training to develop all the skills required for this field. Apart from the degree/diploma and the training, it is important to prepare the right resume for a data science job, and to be well versed with the data science interview questions and answers. Consider our top 100 Data Science Interview Questions and Answers as a starting point for your data scientist interview preparation.
Pruning variable selection ensembles
Zhang, Chunxia, Wu, Yilei, Zhu, Mu
In the context of variable selection, ensemble learning has gained increasing interest due to its great potential to improve selection accuracy and to reduce false discovery rate. A novel ordering-based selective ensemble learning strategy is designed in this paper to obtain smaller but more accurate ensembles. In particular, a greedy sorting strategy is proposed to rearrange the order by which the members are included into the integration process. Through stopping the fusion process early, a smaller subensemble with higher selection accuracy can be obtained. More importantly, the sequential inclusion criterion reveals the fundamental strength-diversity trade-off among ensemble members. By taking stability selection (abbreviated as StabSel) as an example, some experiments are conducted with both simulated and real-world data to examine the performance of the novel algorithm. Experimental results demonstrate that pruned StabSel generally achieves higher selection accuracy and lower false discovery rates than StabSel and several other benchmark methods.
A Flexible Framework for Hypothesis Testing in High-dimensions
Javanmard, Adel, Lee, Jason D.
Hypothesis testing in the linear regression model is a fundamental statistical problem. We consider linear regression in the high-dimensional regime where the number of parameters exceeds the number of samples ($p> n$) and assume that the high-dimensional parameters vector is $s_0$ sparse. We develop a general and flexible $\ell_\infty$ projection statistic for hypothesis testing in this model. Our framework encompasses testing whether the parameter lies in a convex cone, testing the signal strength, testing arbitrary functionals of the parameter, and testing adaptive hypothesis. We show that the proposed procedure controls the type I error under the standard assumption of $s_0 (\log p)/\sqrt{n}\to 0$, and also analyze the power of the procedure. Our numerical experiments confirms our theoretical findings and demonstrate that we control false positive rate (type I error) near the nominal level, and have high power.
A Tube-and-Droplet-based Approach for Representing and Analyzing Motion Trajectories
Lin, Weiyao, Zhou, Yang, Xu, Hongteng, Yan, Junchi, Xu, Mingliang, Wu, Jianxin, Liu, Zicheng
Trajectory analysis is essential in many applications. In this paper, we address the problem of representing motion trajectories in a highly informative way, and consequently utilize it for analyzing trajectories. Our approach first leverages the complete information from given trajectories to construct a thermal transfer field which provides a context-rich way to describe the global motion pattern in a scene. Then, a 3D tube is derived which depicts an input trajectory by integrating its surrounding motion patterns contained in the thermal transfer field. The 3D tube effectively: 1) maintains the movement information of a trajectory, 2) embeds the complete contextual motion pattern around a trajectory, 3) visualizes information about a trajectory in a clear and unified way. We further introduce a droplet-based process. It derives a droplet vector from a 3D tube, so as to characterize the high-dimensional 3D tube information in a simple but effective way. Finally, we apply our tube-and-droplet representation to trajectory analysis applications including trajectory clustering, trajectory classification & abnormality detection, and 3D action recognition. Experimental comparisons with state-of-the-art algorithms demonstrate the effectiveness of our approach.
SOFAR: large-scale association network learning
Uematsu, Yoshimasa, Fan, Yingying, Chen, Kun, Lv, Jinchi, Lin, Wei
Many modern big data applications feature large scale in both numbers of responses and predictors. Better statistical efficiency and scientific insights can be enabled by understanding the large-scale response-predictor association network structures via layers of sparse latent factors ranked by importance. Yet sparsity and orthogonality have been two largely incompatible goals. To accommodate both features, in this paper we suggest the method of sparse orthogonal factor regression (SOFAR) via the sparse singular value decomposition with orthogonality constrained optimization to learn the underlying association networks, with broad applications to both unsupervised and supervised learning tasks such as biclustering with sparse singular value decomposition, sparse principal component analysis, sparse factor analysis, and spare vector autoregression analysis. Exploiting the framework of convexity-assisted nonconvex optimization, we derive nonasymptotic error bounds for the suggested procedure characterizing the theoretical advantages. The statistical guarantees are powered by an efficient SOFAR algorithm with convergence property. Both computational and theoretical advantages of our procedure are demonstrated with several simulation and real data examples.
Exploiting random projections and sparsity with random forests and gradient boosting methods -- Application to multi-label and multi-output learning, random forest model compression and leveraging input sparsity
Within machine learning, the supervised learning field aims at modeling the input-output relationship of a system, from past observations of its behavior. Decision trees characterize the input-output relationship through a series of nested $if-then-else$ questions, the testing nodes, leading to a set of predictions, the leaf nodes. Several of such trees are often combined together for state-of-the-art performance: random forest ensembles average the predictions of randomized decision trees trained independently in parallel, while tree boosting ensembles train decision trees sequentially to refine the predictions made by the previous ones. The emergence of new applications requires scalable supervised learning algorithms in terms of computational power and memory space with respect to the number of inputs, outputs, and observations without sacrificing accuracy. In this thesis, we identify three main areas where decision tree methods could be improved for which we provide and evaluate original algorithmic solutions: (i) learning over high dimensional output spaces, (ii) learning with large sample datasets and stringent memory constraints at prediction time and (iii) learning over high dimensional sparse input spaces.
WWE Payback 2017: Predictions, Match Card For First 'Monday Night Raw' PPV After WrestleMania
WWE's first pay-per-view since WrestleMania 33 is set for Sunday night in San Jose, California. Payback 2017 will feature mostly members of the "Monday Night Raw," roster, though a couple of "SmackDown Live" wrestlers are on the card. WWE Champion Randy Orton is still with the blue brand after the Superstar Shake-up, but he's concluding his feud with Bray Wyatt at Payback. Kevin Owens has appeared on "SmackDown Live" twice since WrestleMania 33, and he'll put his United States Championship on the line against Chris Jericho. Living up to its name, Payback features several rematches.