When training and evaluating machine learning models on a large number of tasks, it is important to not only look at average task accuracy--which may be biased by easy or redundant tasks--but also worst-case accuracy (i.e. the performance on the task with the lowest accuracy). In this work, we show how to use techniques from the distributionally robust optimization (DRO) literature to improve worst-case performance in multitask learning. We highlight several failure cases of DRO when applied off-the-shelf and present an improved method, Lookahead-DRO (L-DRO), which mitigates these issues. The core idea of L-DRO is to anticipate the interaction between tasks during training in order to choose a dynamic re-weighting of the various task losses, which will (i) lead to minimal worst-case loss and (ii) train on as many tasks as possible. After demonstrating the efficacy of L-DRO on a small controlled synthetic setting, we evaluate it on two realistic benchmarks: a multitask version of the CIFAR-100 image classification dataset and a large-scale multilingual language modeling experiment. Our empirical results show that L-DRO achieves a better trade-off between average and worst-case accuracy with little computational overhead compared to several strong baselines. Multitask learning--the process by which a single model is trained to perform a variety of different tasks--has become a subject of increasing interest with many successful applications in a variety of domains (Ruder, 2017; Yu et al., 2020; Wang et al., 2021).

Let's hop on to why we use Transfer Learning! So in Deep Learning, people always go for self-prepared and self-trained models. They always tend to make their models from scratch, which is a good approach. By making a model from scratch, you get complete access to it. You can dictate the play.

LEWES, Del., Oct. 05, 2021 (GLOBE NEWSWIRE) -- John Snow Labs, the Healthcare AI and NLP company and developer of the Spark NLP library, today announced that it will enable free access to its enterprise-grade Annotation Lab and NLP Server software for all users. This announcement comes on the first day of the company's annual NLP Summit, a free online event that brings together the AI community to discuss the most important trends, use cases, and solutions advancing natural language processing (NLP). The Annotation Lab, a robust data labeling and AI/ML solution for teams, enables users to annotate documents, images, and videos. The software automatically trains models using active learning and transfer learning. The simple and efficient project-based workflow helps users leverage real-time analytics on productivity, dataset bias, inter-annotator agreement, and more.

The simple idea of transfer learning is, After Neural Network learned from one task, apply that knowledge to another related task. It is a powerful idea in Deep Learning. You all know in Computer vision and Natural Language Processing tasks required high computational costs and time. So, we can simplify those tasks using Transfer Learning. For example, after we trained a model using images to classify Cars, then that model we can use to recognize other vehicles like trucks.

Transfer learning where the behavior of extracting transferable knowledge from the source domain(s) and reusing this knowledge to target domain has become a research area of great interest in the field of artificial intelligence. Probabilistic graphical models (PGMs) have been recognized as a powerful tool for modeling complex systems with many advantages, e.g., the ability to handle uncertainty and possessing good interpretability. Considering the success of these two aforementioned research areas, it seems natural to apply PGMs to transfer learning. However, although there are already some excellent PGMs specific to transfer learning in the literature, the potential of PGMs for this problem is still grossly underestimated. This paper aims to boost the development of PGMs for transfer learning by 1) examining the pilot studies on PGMs specific to transfer learning, i.e., analyzing and summarizing the existing mechanisms particularly designed for knowledge transfer; 2) discussing examples of real-world transfer problems where existing PGMs have been successfully applied; and 3) exploring several potential research directions on transfer learning using PGM.

I was given Xray baggage scan images by an airport to develop a model that performs automatic detection of dangerous objects (gun and knife). Given only a small amount of Xray images, I am using Domain Adaptation by first collecting a large number of normal (non-Xray) images of dangerous objects from the internet, training a model using only those normal images, then adapting the model to perform well on Xray images. In my previous post, I talked about iterative data collection process for web images of gun and knife to be used for domain adaptation. In this post, I will discuss transfer learning with ResNet50 using the scraped web images. For now, we won't worry about the Xray images and only focus on training the model with the web images. To read this post, it's recommended to have some knowledge about how to apply transfer learning using a model pre-trained on ImageNet in PyTorch. I won't explain every step in detail, but will share some useful tips that can answer questions like:

Very cool library with lots of great ideas on moving toward'green', efficient multimodal machine learning and AI. Kevin Carlberg, AI Research Science Manager at Facebook Reality Labs (quoted from tweet). PyKale is a PyTorch library for multimodal learning and transfer learning with deep learning and dimensionality reduction on graphs, images, texts, and videos. By adopting a unified pipeline-based API design, PyKale enforces standardization and minimalism, via green machine learning concepts of reducing repetitions and redundancy, reusing existing resources, and recycling learning models across areas. PyKale aims to facilitate interdisciplinary, knowledge-aware machine learning research for graphs, images, texts, and videos in applications including bioinformatics, graph analysis, image/video recognition, and medical imaging.

Machine Learning and Artificial Intelligence are being used in many industries and their usage keeps growing with an increase in the performance of the systems respectively. Some of the interesting applications of machine learning are in Self-driving cars and pharmaceutical industries. There are a lot of latest interesting applications being created every day with the use of AI and machine learning. There is a subset of Artificial Intelligence called Deep Learning where there are complex units of neurons that would perform the computations when needed. One thing to note is that when performing these computations, the models would optimize their weights and biases in the process of classifying a certain set of objects in images in the case of image classification tasks.

Brain strain and strain rate are effective in predicting traumatic brain injury (TBI) caused by head impacts. However, state-of-the-art finite element modeling (FEM) demands considerable computational time in the computation, limiting its application in real-time TBI risk monitoring. To accelerate, machine learning head models (MLHMs) were developed, and the model accuracy was found to decrease when the training/test datasets were from different head impacts types. However, the size of dataset for specific impact types may not be enough for model training. To address the computational cost of FEM, the limited strain rate prediction, and the generalizability of MLHMs to on-field datasets, we propose data fusion and transfer learning to develop a series of MLHMs to predict the maximum principal strain (MPS) and maximum principal strain rate (MPSR). We trained and tested the MLHMs on 13,623 head impacts from simulations, American football, mixed martial arts, car crash, and compared against the models trained on only simulations or only on-field impacts. The MLHMs developed with transfer learning are significantly more accurate in estimating MPS and MPSR than other models, with a mean absolute error (MAE) smaller than 0.03 in predicting MPS and smaller than 7 (1/s) in predicting MPSR on all impact datasets. The MLHMs can be applied to various head impact types for rapidly and accurately calculating brain strain and strain rate. Besides the clinical applications in real-time brain strain and strain rate monitoring, this model helps researchers estimate the brain strain and strain rate caused by head impacts more efficiently than FEM.

The limited representation of minorities and disadvantaged populations in large-scale clinical and genomics research has become a barrier to translating precision medicine research into practice. Due to heterogeneity across populations, risk prediction models are often found to be underperformed in these underrepresented populations, and therefore may further exacerbate known health disparities. In this paper, we propose a two-way data integration strategy that integrates heterogeneous data from diverse populations and from multiple healthcare institutions via a federated transfer learning approach. The proposed method can handle the challenging setting where sample sizes from different populations are highly unbalanced. With only a small number of communications across participating sites, the proposed method can achieve performance comparable to the pooled analysis where individual-level data are directly pooled together. We show that the proposed method improves the estimation and prediction accuracy in underrepresented populations, and reduces the gap of model performance across populations. Our theoretical analysis reveals how estimation accuracy is influenced by communication budgets, privacy restrictions, and heterogeneity across populations. We demonstrate the feasibility and validity of our methods through numerical experiments and a real application to a multi-center study, in which we construct polygenic risk prediction models for Type II diabetes in AA population.