Adversarial Regression is a proposition to perform high dimensional non-linear regression with uncertainty estimation. We used Conditional Generative Adversarial Network to obtain an estimate of the full predictive distribution for a new observation. Generative Adversarial Networks (GAN) are implicit generative models which produce samples from a distribution approximating the distribution of the data. The conditional version of it (CGAN) takes the following expression: $\min\limits_G \max\limits_D V(D, G) = \mathbb{E}_{x\sim p_{r}(x)} [log(D(x, y))] + \mathbb{E}_{z\sim p_{z}(z)} [log (1-D(G(z, y)))]$. An approximate solution can be found by training simultaneously two neural networks to model D and G and feeding G with a random noise vector $z$. After training, we have that $G(z, y)\mathrel{\dot\sim} p_{data}(x, y)$. By fixing $y$, we have $G(z|y) \mathrel{\dot\sim} p{data}(x|y)$. By sampling $z$, we can therefore obtain samples following approximately $p(x|y)$, which is the predictive distribution of $x$ for a new $y$. We ran experiments to test various loss functions, data distributions, sample size, size of the noise vector, etc. Even if we observed differences, no experiment outperformed consistently the others. The quality of CGAN for regression relies on fine-tuning a range of hyperparameters. In a broader view, the results show that CGANs are very promising methods to perform uncertainty estimation for high dimensional non-linear regression.

--The performance of machine learning model can be further improved if contextual cues are provided as input along with base features that are directly related to an inference task. In offline learning, one can inspect historical training data to identify contextual clusters either through feature clustering, or handcrafting additional features to describe a context. While offline training enjoys the privilege of learning reliable models based on already-defined contextual features, online training for streaming data may be more challenging-- the data is streamed through time, and the underlying context during a data generation process may change. Furthermore, the problem is exacerbated when the number of possible context is not known. In this study, we propose an online-learning algorithm involving the use of a neural network-based autoencoder to identify contextual changes during training, then compares the currently-inferred context to a knowledge base of learned contexts as training advances. Results show that classifier-training benefits from the automatically discovered contexts which demonstrates quicker learning convergence during contextual changes compared to current methods. Contextual cues can greatly benefit learning of predictive tasks in a machine learning model. A single datapoint may be meaningless.

Figure 4 (left) shows that the probability of being a hit paper increases gradually with career and team novelty, but expedition novelty rises much more quickly as the strongest predictor. Papers involving the most unexpected publication events or conversations are 3.5 times more likely than random to be hit papers. Figure 4 (left) also shows that career and team novelties are highly correlated, suggesting that successful teams not only have members from multiple disciplines, but also members with diverse backgrounds who "glue" interdisciplinary teams together (also see Figure S3). Successful knowledge expeditions, however, are the most likely path associated with breakthrough discovery. When regressing content and context novelties of a paper separately on the three background novelty measures, we find that expedition novelty has by far the largest effect on context novelty (), but team novelty has the marginal top effect on . 2 3, p 0 0 1 β 2 .

This paper investigates the impact of pre-existing offline data on online learning, in the context of dynamic pricing. We study a single-product dynamic pricing problem over a selling horizon of $T$ periods. The demand in each period is determined by the price of the product according to a linear demand model with unknown parameters. We assume that an incumbent price has been tested for $n$ periods in the offline stage before the start of the selling horizon, and the seller has collected $n$ demand observations under the incumbent price from the market. The seller wants to utilize both the pre-existing offline data and the sequential online data to minimize the regret of the online learning process. In the well-separated case where the absolute difference between the incumbent price and the optimal price $\delta$ is lower bounded by a known constant, we prove that the best achievable regret is $\tilde{\Theta}\left(\sqrt{T}\wedge (\frac{T}{n}\vee \log T)\right)$, and show that certain variants of the greedy policy achieve this bound. In the general case where $\delta$ is not necessarily lower bounded by a known constant, we prove that the best achievable regret is $\tilde{\Theta}\left(\sqrt{T}\wedge (\frac{T}{n\delta^2} \vee \frac{\log T}{\delta^2})\right)$, and construct a learning algorithm based on the "optimism in the face of uncertainty" principle, whose regret is optimal up to a logarithm factor. In both cases, our results reveal surprising transformations of the optimal regret rate with respect to the size of offline data, which we refer to as phase transitions. In addition, our result demonstrates that the shape of offline data, measured by $\delta$, also has an intrinsic effect on the optimal regret, and we quantify this effect via the inverse-square law.

We propose a novel discrete-time dynamical system-based framework for achieving adversarial robustness in machine learning models. Our algorithm is originated from robust optimization, which aims to find the saddle point of a min-max optimization problem in the presence of uncertainties. The robust learning problem is formulated as a robust optimization problem, and we introduce a discrete-time algorithm based on a saddle-point dynamical system (SDS) to solve this problem. Under the assumptions that the cost function is convex and uncertainties enter concavely in the robust learning problem, we analytically show that using a diminishing step-size, the stochastic version of our algorithm, SSDS converges asymptotically to the robust optimal solution. The algorithm is deployed for the training of adversarially robust deep neural networks. Although such training involves highly non-convex non-concave robust optimization problems, empirical results show that the algorithm can achieve significant robustness for deep learning. We compare the performance of our SSDS model to other state-of-the-art robust models, e.g., trained using the projected gradient descent (PGD)-training approach. From the empirical results, we find that SSDS training is computationally inexpensive (compared to PGD-training) while achieving comparable performances. SSDS training also helps robust models to maintain a relatively high level of performance for clean data as well as under black-box attacks.

We study the problem of fitting task-specific learning rate schedules from the perspective of hyperparameter optimization. This allows us to explicitly search for schedules that achieve good generalization. We describe the structure of the gradient of a validation error w.r.t. the learning rate, the hypergradient, and based on this we introduce a novel online algorithm. Our method adaptively interpolates between the recently proposed techniques of Franceschi et al. (2017) and Baydin et al. (2017), featuring increased stability and faster convergence. We show empirically that the proposed method compares favourably with baselines and related methods in terms of final test accuracy.

In this paper, we propose a hierarchical feature-aware tracking framework for efficient visual tracking. Recent years, ensembled trackers which combine multiple component trackers have achieved impressive performance. In ensembled trackers, the decision of results is usually a post-event process, i.e., tracking result for each tracker is first obtained and then the suitable one is selected according to result ensemble. In this paper, we propose a pre-event method. We construct an expert pool with each expert being one set of features. For each frame, several experts are first selected in the pool according to their past performance and then they are used to predict the object. The selection rate of each expert in the pool is then updated and tracking result is obtained according to result ensemble. We propose a novel pre-known expert-adaptive selection strategy. Since the process is more efficient, more experts can be constructed by fusing more types of features which leads to more robustness. Moreover, with the novel expert selection strategy, overfitting caused by fixed experts for each frame can be mitigated. Experiments on several public available datasets demonstrate the superiority of the proposed method and its state-of-the-art performance among ensembled trackers.

Artificial neural networks have exceeded human-level performance in accomplishing several individual tasks (e.g. voice recognition, object recognition, and video games). However, such success remains modest compared to human intelligence that can learn and perform an unlimited number of tasks. Humans' ability of learning and accumulating knowledge over their lifetime is an essential aspect of their intelligence. Continual machine learning aims at a higher level of machine intelligence through providing the artificial agents with the ability to learn online from a non-stationary and never-ending stream of data. A key component of such a never-ending learning process is to overcome the catastrophic forgetting of previously seen data, a problem that neural networks are well known to suffer from. The work described in this thesis has been dedicated to the investigation of continual learning and solutions to mitigate the forgetting phenomena in neural networks. To approach the continual learning problem, we first assume a task incremental setting where tasks are received one at a time and data from previous tasks are not stored. Since the task incremental setting can't be assumed in all continual learning scenarios, we also study the more general online continual setting. We consider an infinite stream of data drawn from a non-stationary distribution with a supervisory or self-supervisory training signal. The proposed methods in this thesis have tackled important aspects of continual learning. They were evaluated on different benchmarks and over various learning sequences. Advances in the state of the art of continual learning have been shown and challenges for bringing continual learning into application were critically identified.

Remember the day when Steve Jobs announced the very first iPhone? Two important things happened that day. Number one, the world was getting a first glimpse at a new technology that was like no other: being able to touch your phone and therefore have the entire world at the tips of your fingers. Number two, everybody was certain this new technology would take over and somehow rule the planet in the next few years. Fortunately, we're still controlling our phones and they've not taken over the Earth for now.

Wyebot has raised $2.5 million through Series-Seed funding, and announced broad availability of its sensor-based Wi-Fi assurance platform for enterprise and educational facilities. Innospark Ventures and Tectonic Ventures led the funding round, which is enabling Wyebot to scale its go-to-market team and expand machine learning activities. The company's vendor agnostic Wireless Intelligence Platform (WIP) can be up and running in about five minutes, according to a spokesperson, and uses AI-powered algorithms and sensors to prevent and help fix Wi-Fi network issues, both before and as they happen. "Even though Wi-Fi is seemingly everywhere today, the necessary expertise and skillset to properly manage wireless networks is not as omnipresent," said Venkat Srinivasan, managing director and founder of Innospark Ventures, in a statement. That's why we're so excited about Wyebot and its WIP – the AI capabilities and plug-and-play ease of use make it the ideal solution for any organization seeking WiFi assurance." Wyebot's on-premise sensor hardware collects RF data, both Wi-Fi and non-Wi-Fi, using four Wi-Fi radios. Sensors then send metadata to a cloud platform where WIP's AI engine employs predictive analytics to detect problems and automatically recommend solutions to help keep Wi-Fi networks up and running. Each sensor covers 10,000 square feet. According to Wyebot, WIP reduces mean time to resolution by up to 90%, decreases Wi-Fi problem tickets by 50%, and reduces remote site visits by 80%, according to Wyebot. In enterprise networks, the platform provides visibility and optimization in environments where networks can get stressed by an increasing number of devices, including personal handsets and watches and facility devices like printers and smart thermostats and lighting. Wyebot also stressed the benefits of the WIP platform in distributed environments, as more companies have employees who work remotely. "Today's business need for constant connectivity has created a burden on IT departments to keep an entire organization connected – especially difficult with the proliferation of distributed enterprises and campuses," said Roger Sands, CEO, Wyebot, in a statement. "Organizations can't afford the lost productivity that occurs when their WiFi isn't working or they are unable to access critical applications and information in the cloud.