For samples generated by stationary and φ-mixing processes we give generalization guarantees uniform over Lipschitz, smooth or unconstrained loss classes. The result depends on an empirical estimator of the distance of data drawn from the invariant distribution to the sample path. The mixing time (the time needed to obtain approximate independence) enters these results explicitely only in an additive way. For slowly mixing processes this can be a considerable advantage over results with multiplicative dependence on the mixing time. Because of the applicability to unconstrained loss classes, where the bound depends only on local Lipschitz properties at the sample points, it may be interesting also for iid processes, whenever the data distribution is a sufficiently simple object.

Unsupervised domain adaptation is the problem setting where data generating distributions in the source and target domains are different, and labels in the target domain are unavailable. One important question in unsupervised domain adaptation is how to measure the difference between the source and target domains. A previously proposed discrepancy that does not use the source domain labels requires high computational cost to estimate and may lead to a loose generalization error bound in the target domain. To mitigate these problems, we propose a novel discrepancy called source-guided discrepancy ($S$-disc), which exploits labels in the source domain. As a consequence, $S$-disc can be computed efficiently with a finite sample convergence guarantee. In addition, we show that $S$-disc can provide a tighter generalization error bound than the one based on an existing discrepancy. Finally, we report experimental results that demonstrate the advantages of $S$-disc over the existing discrepancies.

Feature engineering is a crucial step in the process of predictive modeling. It involves the transformation of given feature space, typically using mathematical functions, with the objective of reducing the modeling error for a given target. However, there is no well-defined basis for performing effective feature engineering. It involves domain knowledge, intuition, and most of all, a lengthy process of trial and error. The human attention involved in overseeing this process significantly influences the cost of model generation. We present a new framework to automate feature engineering. It is based on performance driven exploration of a transformation graph, which systematically and compactly enumerates the space of given options. A highly efficient exploration strategy is derived through reinforcement learning on past examples.