Deep Learning
Weak Adaptation Learning -- Addressing Cross-domain Data Insufficiency with Weak Annotator
Xu, Shichao, Wang, Lixu, Wang, Yixuan, Zhu, Qi
Data quantity and quality are crucial factors for data-driven learning methods. In some target problem domains, there are not many data samples available, which could significantly hinder the learning process. While data from similar domains may be leveraged to help through domain adaptation, obtaining high-quality labeled data for those source domains themselves could be difficult or costly. To address such challenges on data insufficiency for classification problem in a target domain, we propose a weak adaptation learning (WAL) approach that leverages unlabeled data from a similar source domain, a low-cost weak annotator that produces labels based on task-specific heuristics, labeling rules, or other methods (albeit with inaccuracy), and a small amount of labeled data in the target domain. Our approach first conducts a theoretical analysis on the error bound of the trained classifier with respect to the data quantity and the performance of the weak annotator, and then introduces a multi-stage weak adaptation learning method to learn an accurate classifier by lowering the error bound. Our experiments demonstrate the effectiveness of our approach in learning an accurate classifier with limited labeled data in the target domain and unlabeled data in the source domain.
Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm
Reynolds, Laria, McDonell, Kyle
Prevailing methods for mapping large generative language models to supervised tasks may fail to sufficiently probe models' novel capabilities. Using GPT-3 as a case study, we show that 0-shot prompts can significantly outperform few-shot prompts. We suggest that the function of few-shot examples in these cases is better described as locating an already learned task rather than meta-learning. This analysis motivates rethinking the role of prompts in controlling and evaluating powerful language models. In this work, we discuss methods of prompt programming, emphasizing the usefulness of considering prompts through the lens of natural language. We explore techniques for exploiting the capacity of narratives and cultural anchors to encode nuanced intentions and techniques for encouraging deconstruction of a problem into components before producing a verdict. Informed by this more encompassing theory of prompt programming, we also introduce the idea of a metaprompt that seeds the model to generate its own natural language prompts for a range of tasks. Finally, we discuss how these more general methods of interacting with language models can be incorporated into existing and future benchmarks and practical applications.
Understanding Emails and Drafting Responses -- An Approach Using GPT-3
Thiergart, Jonas, Huber, Stefan, รbellacker, Thomas
Providing computer systems with the ability to understand and generate natural language has long been a challenge of engineers. Recent progress in natural language processing (NLP), like the GPT-3 language model released by OpenAI, has made both possible to an extent. In this paper, we explore the possibility of rationalising email communication using GPT-3. First, we demonstrate the technical feasibility of understanding incoming emails and generating responses, drawing on literature from the disciplines of software engineering as well as data science. Second, we apply knowledge from both business studies and, again, software engineering to identify ways to tackle challenges we encountered. Third, we argue for the economic viability of such a solution by analysing costs and market demand. We conclude that applying GPT-3 to rationalising email communication is feasible both technically and economically.
Graph Neural Network for Traffic Forecasting: A Survey
Traffic forecasting is important for the success of intelligent transportation systems. Deep learning models, including convolution neural networks and recurrent neural networks, have been extensively applied in traffic forecasting problems to model spatial and temporal dependencies. In recent years, to model the graph structures in transportation systems as well as contextual information, graph neural networks have been introduced and have achieved state-of-the-art performance in a series of traffic forecasting problems. In this survey, we review the rapidly growing body of research using different graph neural networks, e.g. graph convolutional and graph attention networks, in various traffic forecasting problems, e.g. road traffic flow and speed forecasting, passenger flow forecasting in urban rail transit systems, and demand forecasting in ride-hailing platforms. We also present a comprehensive list of open data and source resources for each problem and identify future research directions. To the best of our knowledge, this paper is the first comprehensive survey that explores the application of graph neural networks for traffic forecasting problems. We have also created a public GitHub repository where the latest papers, open data, and source resources will be updated.
Top Machine Learning Books to read in 2021
Machine Learning is on the raise with big businesses implementing AI in every single business domain. If you want to jump in, or you're already in trying to learn more, here's the list of top machine learning books you can find in 2021. Whether you're an absolute beginner and haven't heard of Python before (it's not a snake!), or you've dozens of CNNs in PyTorch, you'll find something for you. Introduction to Machine Learning with Python is a gentle introduction to all the essential tasks required to execute machine learning. It does not need previous knowledge of Python or any knowledge about machine learning. Instead, it incorporates basic principles in Python and addresses different approaches in Python.
Python AI Machine Learning, OpenCV
Python AI Machine Learning, OpenCV Start your career path in Python Artificial Intelligence Machine Learning now!! What you'll learn Python required for AI, Machine Learning & Data Science 2021 Ready to explore machine learning and artificial intelligence in python? This python Artificial Intelligence machine learning and OpenCV course (A-Z) contains 5 different series designed to teach you the ins and outs of Machine Learning and Artificial intelligence. It talks about fundamental Machine Learning algorithms, neural networks, Deep Learning, OpenCV and finally developing an Artificial Intelligence that can play the game of Flappy Bird.
Deep Learning Prerequisites: Logistic Regression in Python
Data science techniques for professionals and students - learn the theory behind logistic regression and code in Python Created by Lazy Programmer Inc. English [Auto-generated], Portuguese [Auto-generated], 1 more Created by Lazy Programmer Inc. English [Auto-generated], Portuguese [Auto-generated], 1 more Created by Lazy Programmer Inc.
10 Artificial Intelligence Predictions for 2021 - CLOUDit-eg
The arrival of the New Year brings us to think in many areas. What does 2021 hold in store for artificial intelligence? Here are 10 Artificial Intelligence predictions, from academic research to capital markets to regulation. We will take stock in December 2021 to assess the results. Autonomous vehicle developers like Waymo and Cruise have ongoing and massive cash flow needs.
Evolving Neural Networks in JAX
"So why should I switch from insert-autodiff-library to JAX?". Here is my answer: JAX is not simply a fast library for automatic differentiation. If your scientific computing project wants to benefit from XLA, JIT-compilation and the bulk-array programming paradigm -- then JAX provides a wonderful API. While PyTorch relies on pre-compiled kernels and fast C code for most common Deep Learning applications, JAX allows us to leverage a high-level interface for programming your favorite accelerators. But this is not restricted to standard gradient-based optimization setups.