Adversarial Regression is a proposition to perform high dimensional non-linear regression with uncertainty estimation. We used Conditional Generative Adversarial Network to obtain an estimate of the full predictive distribution for a new observation. Generative Adversarial Networks (GAN) are implicit generative models which produce samples from a distribution approximating the distribution of the data. The conditional version of it (CGAN) takes the following expression: $\min\limits_G \max\limits_D V(D, G) = \mathbb{E}_{x\sim p_{r}(x)} [log(D(x, y))] + \mathbb{E}_{z\sim p_{z}(z)} [log (1-D(G(z, y)))]$. An approximate solution can be found by training simultaneously two neural networks to model D and G and feeding G with a random noise vector $z$. After training, we have that $G(z, y)\mathrel{\dot\sim} p_{data}(x, y)$. By fixing $y$, we have $G(z|y) \mathrel{\dot\sim} p{data}(x|y)$. By sampling $z$, we can therefore obtain samples following approximately $p(x|y)$, which is the predictive distribution of $x$ for a new $y$. We ran experiments to test various loss functions, data distributions, sample size, size of the noise vector, etc. Even if we observed differences, no experiment outperformed consistently the others. The quality of CGAN for regression relies on fine-tuning a range of hyperparameters. In a broader view, the results show that CGANs are very promising methods to perform uncertainty estimation for high dimensional non-linear regression.

Real-world problems typically require the simultaneous op timization of several, often conflicting objectives. Many of these multi-objective optimization problems are characterized by wide ranges of uncertainties in their deci sion variables or objective functions, which further increases the complexity of optim ization. To cope with such uncertainties, robust optimization is widely studied aiming to distinguish candidate solutions with uncertain objectives specified by confidence intervals, probability distributions or sampled data. However, existing techniques most ly either fail to consider the actual distributions or assume uncertainty as instance s of uniform or Gaussian distributions. This paper introduces an empirical approac h that enables an efficient comparison of candidate solutions with uncertain objectiv es that can follow arbitrary distributions. Given two candidate solutions under compar ison, this operator calculates the probability that one solution dominates the other in terms of each uncertain objective. It can substitute for the standard comparison op erator of existing optimization techniques such as evolutionary algorithms to ena ble discovering robust solutions to problems with multiple uncertain objectives. Th is paper also proposes to incorporate various uncertainties in well-known multi-ob jective problems to provide a benchmark for evaluating uncertainty-aware optimization techniques. The proposed comparison operator and benchmark suite are integrated int o an existing optimization tool that features a selection of multi-objective optimiza tion problems and algorithms. Experiments show that in comparison with existing techniqu es, the proposed approach achieves higher optimization quality at lower overheads. The authors are with the Department of Computer Science, Friedr ich-Alexander-Universit at Erlangen-N urnberg (FAU), Erlangen 91058, Germany. A a candidate solution B a candidate solution B beta distribution C comparison operator c positive constant E expected value e approximation error f objective function N Gaussian distribution N number of samples or quantile cuts n number of decision variables m number of objective functions S sequence of samples s sample from an uncertain objective's distribution U uniform distribution u uncertainty added to an optimization problem V ar variance X random variable x decision variable γ comparison threshold δ tolerance (bound on an error) σ standard deviation ω interval width in a histogram 2 1 Introduction Real-world problems typically demand solutions that are optimized with respect to multiple criteria called objectives. In these so-called multi-objective optimization problems, the objectives often conflict with each other such that no single solution can b e found to be optimal in all objectives. Instead, one usually searches for a set of non-d ominated solutions known as Pareto front or Pareto set that provide decent tradeoffs among objectives.

While deep learning has been incredibly successful in modeling tasks with large, carefully curated labeled datasets, its application to problems with limited labeled data remains a challenge. The aim of the present work is to improve the label efficiency of large neural networks operating on audio data through a combination of multitask learning and self-supervised learning on unlabeled data. We trained an end-to-end audio feature extractor based on WaveNet that feeds into simple, yet versatile task-specific neural networks. We describe several easily implemented self-supervised learning tasks that can operate on any large, unlabeled audio corpus. We demonstrate that, in scenarios with limited labeled training data, one can significantly improve the performance of three different supervised classification tasks individually by up to 6% through simultaneous training with these additional self-supervised tasks. We also show that incorporating data augmentation into our multitask setting leads to even further gains in performance.

--The performance of machine learning model can be further improved if contextual cues are provided as input along with base features that are directly related to an inference task. In offline learning, one can inspect historical training data to identify contextual clusters either through feature clustering, or handcrafting additional features to describe a context. While offline training enjoys the privilege of learning reliable models based on already-defined contextual features, online training for streaming data may be more challenging-- the data is streamed through time, and the underlying context during a data generation process may change. Furthermore, the problem is exacerbated when the number of possible context is not known. In this study, we propose an online-learning algorithm involving the use of a neural network-based autoencoder to identify contextual changes during training, then compares the currently-inferred context to a knowledge base of learned contexts as training advances. Results show that classifier-training benefits from the automatically discovered contexts which demonstrates quicker learning convergence during contextual changes compared to current methods. Contextual cues can greatly benefit learning of predictive tasks in a machine learning model. A single datapoint may be meaningless.

Estimator Vectors: OOV Word Embeddings based on Subword and Context Clue Estimates Raj Patel Carlotta Domeniconi † Abstract Semantic representations of words have been successfully extracted from unlabeled corpuses using neural network models like word2vec. These representations are generally high quality and are computationally inexpensive to train, making them popular. However, these approaches generally fail to approximate out of vocabulary (OOV) words, a task humans can do quite easily, using word roots and context clues. This paper proposes a neural network model that learns high quality word representations, subword representations, and context clue representations jointly. Learning all three types of representations together enhances the learning of each, leading to enriched word vectors, along with strong estimates for OOV words, via the combination of the corresponding context clue and subword embeddings. Our model, called Estimator Vectors (EV), learns strong word embed-dings and is competitive with state of the art methods for OOV estimation. 1 Introduction Semantic representations of words are useful for many natural language processing (NLP) tasks. While there exists many ways to learn them, models like word2vec [11] and GloVe [15] have been shown to be very efficient at producing high quality word embeddings. These embeddings not only capture similarity between words, but also capture some algebraic relationships between words. These models, though, also have some downsides. One major drawback is that they can only learn embeddings for words in the vocabulary, determined by the corpus they were trained on.

Figure 4 (left) shows that the probability of being a hit paper increases gradually with career and team novelty, but expedition novelty rises much more quickly as the strongest predictor. Papers involving the most unexpected publication events or conversations are 3.5 times more likely than random to be hit papers. Figure 4 (left) also shows that career and team novelties are highly correlated, suggesting that successful teams not only have members from multiple disciplines, but also members with diverse backgrounds who "glue" interdisciplinary teams together (also see Figure S3). Successful knowledge expeditions, however, are the most likely path associated with breakthrough discovery. When regressing content and context novelties of a paper separately on the three background novelty measures, we find that expedition novelty has by far the largest effect on context novelty (), but team novelty has the marginal top effect on . 2 3, p 0 0 1 β 2 .

Discovering content and stories in movies is one of the most important concepts in multimedia content research studies. Network models have proven to be an efficient choice for this purpose. When an audience watches a movie, they usually compare the characters and the relationships between them. For this reason, most of the models developed so far are based on social networks analysis. They focus essentially on the characters at play. By analyzing characters' interactions, we can obtain a broad picture of the narration's content. Other works have proposed to exploit semantic elements such as scenes, dialogues, etc. However, they are always captured from a single facet. Motivated by these limitations, we introduce in this work a multilayer network model to capture the narration of a movie based on its script, its subtitles, and the movie content. After introducing the model and the extraction process from the raw data, we perform a comparative analysis of the whole 6-movie cycle of the Star Wars saga. Results demonstrate the effectiveness of the proposed framework for video content representation and analysis.

This paper investigates the impact of pre-existing offline data on online learning, in the context of dynamic pricing. We study a single-product dynamic pricing problem over a selling horizon of $T$ periods. The demand in each period is determined by the price of the product according to a linear demand model with unknown parameters. We assume that an incumbent price has been tested for $n$ periods in the offline stage before the start of the selling horizon, and the seller has collected $n$ demand observations under the incumbent price from the market. The seller wants to utilize both the pre-existing offline data and the sequential online data to minimize the regret of the online learning process. In the well-separated case where the absolute difference between the incumbent price and the optimal price $\delta$ is lower bounded by a known constant, we prove that the best achievable regret is $\tilde{\Theta}\left(\sqrt{T}\wedge (\frac{T}{n}\vee \log T)\right)$, and show that certain variants of the greedy policy achieve this bound. In the general case where $\delta$ is not necessarily lower bounded by a known constant, we prove that the best achievable regret is $\tilde{\Theta}\left(\sqrt{T}\wedge (\frac{T}{n\delta^2} \vee \frac{\log T}{\delta^2})\right)$, and construct a learning algorithm based on the "optimism in the face of uncertainty" principle, whose regret is optimal up to a logarithm factor. In both cases, our results reveal surprising transformations of the optimal regret rate with respect to the size of offline data, which we refer to as phase transitions. In addition, our result demonstrates that the shape of offline data, measured by $\delta$, also has an intrinsic effect on the optimal regret, and we quantify this effect via the inverse-square law.

The results of data mining endeavors are majorly driven by data quality. Throughout these deployments, serious show-stopper problems are still unresolved, such as: data collection ambiguities, data imbalance, hidden biases in data, the lack of domain information, and data incompleteness. This paper is based on the premise that context can aid in mitigating these issues. In a traditional data science lifecycle, context is not considered. Context-driven Data Science Lifecycle (C-DSL); the main contribution of this paper, is developed to address these challenges. Two case studies (using data-sets from sports events) are developed to test C-DSL. Results from both case studies are evaluated using common data mining metrics such as: coefficient of determination (R2 value) and confusion matrices. The work presented in this paper aims to re-define the lifecycle and introduce tangible improvements to its outcomes.

Neural Architecture Search (NAS) represents a class of methods to generate the optimal neural network architecture and typically iterate over candidate architectures till convergence over some particular metric like validation loss. They are constrained by the available computation resources, especially in enterprise environments. In this paper, we propose a new approach for NAS, called NASIB, which adapts and attunes to the computation resources (budget) available by varying the exploration vs. exploitation trade-off. We reduce the expert bias by searching over an augmented search space induced by Superkernels. The proposed method can provide the architecture search useful for different computation resources and different domains beyond image classification of natural images where we lack bespoke architecture motifs and domain expertise. We show, on CIFAR10, that itis possible to search over a space that comprises of 12x more candidate operations than the traditional prior art in just 1.5 GPU days, while reaching close to state of the art accuracy. While our method searches over an exponentially larger search space, it could lead to novel architectures that require lesser domain expertise, compared to the majority of the existing methods.