Despite continuing medical advances, the rate of newborn morbidity and mortality globally remains high, with over 6 million casualties every year. The prediction of pathologies affecting newborns based on their cry is thus of significant clinical interest, as it would facilitate the development of accessible, low-cost diagnostic tools\cut{ based on wearables and smartphones}. However, the inadequacy of clinically annotated datasets of infant cries limits progress on this task. This study explores a neural transfer learning approach to developing accurate and robust models for identifying infants that have suffered from perinatal asphyxia. In particular, we explore the hypothesis that representations learned from adult speech could inform and improve performance of models developed on infant speech. Our experiments show that models based on such representation transfer are resilient to different types and degrees of noise, as well as to signal loss in time and frequency domains.

The introduction of deep learning and transfer learning techniques in fields such as computer vision allowed a leap forward in the accuracy of image classification tasks. Currently there is only limited use of such techniques in neuroscience. The challenge of using deep learning methods to successfully train models in neuroscience, lies in the complexity of the information that is processed, the availability of data and the cost of producing sufficient high quality annotations. Inspired by its application in computer vision, we introduce transfer learning on electrophysiological data to enable training a model with limited amounts of data. Our method was tested on the dataset of the BCI competition IV 2a and compared to the top results that were obtained using traditional machine learning techniques. Using our DL model we outperform the top result of the competition by 33%. We also explore transferability of knowledge between trained models over different experiments, called inter-experimental transfer learning. This reduces the amount of required data even further and is especially useful when few subjects are available. This method is able to outperform the standard deep learning methods used in the BCI competition IV 2b approaches by 18%. In this project we propose a method that can produce reliable electroencephalography (EEG) signal classification, based on modest amounts of training data through the use of transfer learning.

Parameter inefficiency, in the context of transfer learning for NLP, arises when an entirely new model needs to be trained for every downstream task and the number of parameters grows too large. A recent paper proposes adapter modules which provide parameter efficiency by only adding a few trainable parameters per task, and as new tasks are added previous ones don't require revisiting. The main idea of this paper is to enable transfer learning for NLP on an incoming stream of tasks without training a new model for every new task. A standard fine-tuning model copies weights from a pre-trained network and tunes them on a downstream task which requires a new set of weights for each task. In other words, the parameters are adjusted together with new layers for each task.

We propose an approach for transfer learning with GAN architectures. In general, transfer learning enables deep networks for classification tasks to be trained with limited computing and data resources. However a similar approach is missing in the specific context of generative tasks. This is partly due to the fact that the extremal layers of the two networks of a GAN, which should be learned during transfer, are on two opposite sides. This requires back-propagating information through both networks, which is computationally expensive. We develop a method to directly train these extremal layers against each other, by-passing all the intermediate layers. We also prove rigorously, for Wasserstein GANs, a theorem ensuring the convergence of the learning of the transferred GAN. Finally, we compare our method to state-of-the-art methods and show that our method converges much faster and requires less data.

We present a transfer learning strategy which fundamentally relies on Bayesian deep learning and the ability to represent Exploration and adaptation to new tasks in a transfer a distribution over functions, as in (Bachman et al., 2018) learning setup is a central challenge in reinforcement (Garnelo et al., 2018). Bayesian methods rely on modeling learning. In this work, we build on the uncertainty over value functions to represent the agent's the idea of modeling a distribution over policies belief of the environment. Recent work has shown that in a Bayesian deep reinforcement learning setup neural networks can be used to represent an uncertainty to propose a transfer strategy. Recent works over the space of all possible functions (Bachman et al., have shown to induce diversity in the learned 2018). The idea of modeling a distribution over functions policies by maximizing the entropy of a distribution can be adapted in the RL setting to model a distribution over of policies (Bachman et al., 2018; Garnelo policies, such that we can also maximize the entropy over et al., 2018) and thus, we postulate that our this distribution of policies. This is similar to maximum proposed approach leads to faster exploration resulting entropy exploration in RL, where instead of local entropy in improved transfer learning. We support maximization, recent work maximizes the global entropy our hypothesis by demonstrating favorable over the space of all possible sub-optimal policies.

In this work, we evaluate adversarial robustness in the context of transfer learning from a source trained on CIFAR 100 to a target network trained on CIFAR 10. Specifically, we study the effects of using robust optimisation in the source and target networks. This allows us to identify transfer learning strategies under which adversarial defences are successfully retained, in addition to revealing potential vulnerabilities. We study the extent to which features learnt by a fast gradient sign method (FGSM) and its iterative alternative (PGD) can preserve their defence properties against black and white-box attacks under three different transfer learning strategies. We find that using PGD examples during training on the source task leads to more general robust features that are easier to transfer. Furthermore, under successful transfer, it achieves 5.2% more accuracy against white-box PGD attacks than suitable baselines. Overall, our empirical evaluations give insights on how well adversarial robustness under transfer learning can generalise.

Human learners have the natural ability to use knowledge gained in one setting for learning in a different but related setting. This ability to transfer knowledge from one task to another is essential for effective learning. In this paper, we study transfer learning in the context of nonparametric classification based on observations from different distributions under the posterior drift model, which is a general framework and arises in many practical problems. We first establish the minimax rate of convergence and construct a rate-optimal two-sample weighted $K$-NN classifier. The results characterize precisely the contribution of the observations from the source distribution to the classification task under the target distribution. A data-driven adaptive classifier is then proposed and is shown to simultaneously attain within a logarithmic factor of the optimal rate over a large collection of parameter spaces. Simulation studies and real data applications are carried out where the numerical results further illustrate the theoretical analysis. Extensions to the case of multiple source distributions are also considered.

Transfer learning aims at building robust prediction models by transferring knowledge gained from one problem to another. In the semantic Web, learning tasks are enhanced with semantic representations. We exploit their semantics to augment transfer learning by dealing with when to transfer with semantic measurements and what to transfer with semantic embeddings. We further present a general framework that integrates the above measurements and embeddings with existing transfer learning algorithms for higher performance. It has demonstrated to be robust in two real-world applications: bus delay forecasting and air quality forecasting.

In this tutorial, you will learn how to perform transfer learning with Keras, Deep Learning, and Python on your own custom datasets. You're just hired by Yelp to work in their computer vision department. Yelp has just launched a new feature on its website that allows reviewers to take photos of their food/dishes and then associate them with particular items on a restaurant's menu. Certain nefarious users aren't taking photos of their dishes…instead, they are taking photos of… (well, you can probably guess). Figure out how to create an automated computer vision application that can distinguish between "food" and "not food", thereby allowing Yelp to continue with their new feature launch and provide value to their users. So, how are you going to build such an application? The answer lies in transfer learning via deep learning. Today marks the start of a brand new set of tutorials on transfer learning using Keras.

In this tutorial, you will learn how to perform transfer learning with Keras, Deep Learning, and Python on your own custom datasets. You're just hired by Yelp to work in their computer vision department. Yelp has just launched a new feature on its website that allows reviewers to take photos of their food/dishes and then associate them with particular items on a restaurant's menu. Certain nefarious users aren't taking photos of their dishes…instead, they are taking photos of… (well, you can probably guess). Figure out how to create an automated computer vision application that can distinguish between "food" and "not food", thereby allowing Yelp to continue with their new feature launch and provide value to their users. So, how are you going to build such an application? The answer lies in transfer learning via deep learning. Today marks the start of a brand new set of tutorials on transfer learning using Keras.