Deep Learning
Can GPT-3 write misinformation? Yup, it sure can
It's the worry that creeps in whenever people write about GPT-3: could this be used for bad? We've covered the technological advances in AI text generation like those from OpenAI a lot. There are always the "oohs" and "aah" about what it can do (write a self-help blog, for instance) and then others pointing out what it can't. But in the background is the question of how the ability to instantaneously "write" large amounts of text based on certain prompts could change the internet in unstoppable ways. Can artificial intelligence like GPT-3 be used for something like misinformation?
Deep Learning in Machine Vision Market: Opportunities, Demand and Forecasts, 2021-2027
Furthermore, it provides measures to subdue the threats that have impacted the business sphere. The report also entails in-depth information pertaining to future market prospects and current trends to help business holders in formulating strategies that can boost the profit margins in the forthcoming years. Proceeding further, the document includes crucial insights pertaining to growth rate and market share projections of each market segment. Key companies influencing the industry trends have been examined thoroughly as well. Moreover, the report studies the impact of the covid-19 pandemic on business dynamics globally and regionally to impart a better understanding of growth prospects over the analysis timeframe.
How artificial intelligence might disrupt diagnostics in hematology in the near future
Artificial intelligence (AI) is about to make itself indispensable in the health care sector. Examples of successful applications or promising approaches range from the application of pattern recognition software to pre-process and analyze digital medical images, to deep learning algorithms for subtype or disease classification, and digital twin technology and in silico clinical trials. Moreover, machine-learning techniques are used to identify patterns and anomalies in electronic health records and to perform ad-hoc evaluations of gathered data from wearable health tracking devices for deep longitudinal phenotyping. In the last years, substantial progress has been made in automated image classification, reaching even superhuman level in some instances. Despite the increasing awareness of the importance of the genetic context, the diagnosis in hematology is still mainly based on the evaluation of the phenotype. Either by the analysis of microscopic images of cells in cytomorphology or by the analysis of cell populations in bidimensional plots obtained by flow cytometry. Here, AI algorithms not only spot details that might escape the human eye, but might also identify entirely new ways of interpreting these images. With the introduction of high-throughput next-generation sequencing in molecular genetics, the amount of available information is increasing exponentially, priming the field for the application of machine learning approaches. The goal of all the approaches is to allow personalized and informed interventions, to enhance treatment success, to improve the timeliness and accuracy of diagnoses, and to minimize technically induced misclassifications. The potential of AI-based applications is virtually endless but where do we stand in hematology and how far can we go?
3 AI startups revolutionizing NLP
Natural language processing (NLP) has been a long-standing dream of computer scientists that dates back to the days of ELIZA and even to the fundamental foundations of computing itself (Turing Test, anybody?). NLP has undergone a dramatic revolution in the past few years, with the statistical methods of the past giving way to approaches based on deep learning, or neural networks. Applying deep learning to NLP has led to massive, sophisticated, general purpose language models, like GPT-3, capable of generating text that is truly indistinguishable from human writing. GPT-3, for example, unlocks features such as those found in Microsoft's new "no-code" Power Apps platform, where you can enter a natural language description of a query, and the back end will generate the code (a Power Fx expression based on Excel syntax). NLP has vast potential across the enterprise, and it's not just the giants like Google or Microsoft that are bringing products to the table.
How to build your own meme generator with machine learning
In this article, I'll show you how I built a system called AI-Memer that generates memes using the latest AI models. I start with a high-level description of the system components before getting into the background of memes and details of the components. I'll then show you how to generate your own memes using the Google Colab, here. After a brief discussion of results and next steps, you can see some sample memes in the appendix. Oh, and I'll show a newly generated meme at the head of each section.
Deep Learning on Graphs for Natural Language Processing - Facebook Research
This tutorial of Deep Learning on Graphs for Natural Language Processing (DLG4NLP) is timely for the computational linguistics community, and covers relevant and interesting topics, including automatic graph construction for NLP, graph representation learning for NLP, various advanced GNN based models (e.g., graph2seq, graph2tree, and graph2graph) for NLP, and the applications of GNNs in various NLP tasks (e.g., machine translation, natural language generation, information extraction and semantic parsing). The intended audiences for this tutorial mainly include graduate students and researchers in the field of Natural Language Processing and industry professionals who want to know how the state-of-the-art deep learning on graphs techniques can help solve important yet challenging Natural Language Processing problems.
On-the-Fly Aligned Data Augmentation for Sequence-to-Sequence ASR
Lam, Tsz Kin, Ohta, Mayumi, Schamoni, Shigehiko, Riezler, Stefan
We propose an on-the-fly data augmentation method for automatic speech recognition (ASR) that uses alignment information to generate effective training samples. Our method, called Aligned Data Augmentation (ADA) for ASR, replaces transcribed tokens and the speech representations in an aligned manner to generate previously unseen training pairs. The speech representations are sampled from an audio dictionary that has been extracted from the training corpus and inject speaker variations into the training examples. The transcribed tokens are either predicted by a language model such that the augmented data pairs are semantically close to the original data, or randomly sampled. Both strategies result in training pairs that improve robustness in ASR training. Our experiments on a Seq-to-Seq architecture show that ADA can be applied on top of SpecAugment, and achieves about 9-23% and 4-15% relative improvements in WER over SpecAugment alone on LibriSpeech 100h and LibriSpeech 960h test datasets, respectively.
Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks
Tolan, Songรผl | Pesole, Annarosa (Joint Research Centre, European Commission) | Martรญnez-Plumed, Fernando (Joint Research Centre, European Commission) | Fernรกndez-Macรญas, Enrique | Hernรกndez-Orallo, Josรฉ (Universitat Politรจcnica de Valรจncia) | Gรณmez, Emilia (Leverhulme Centre for the Future of Intelligence)
In this paper we develop a framework for analysing the impact of Artificial Intelligence (AI) on occupations. This framework maps 59 generic tasks from worker surveys and an occupational database to 14 cognitive abilities (that we extract from the cognitive science literature) and these to a comprehensive list of 328 AI benchmarks used to evaluate research intensity across a broad range of different AI areas. The use of cognitive abilities as an intermediate layer, instead of mapping work tasks to AI benchmarks directly, allows for an identification of potential AI exposure for tasks for which AI applications have not been explicitly created. An application of our framework to occupational databases gives insights into the abilities through which AI is most likely to affect jobs and allows for a ranking of occupations with respect to AI exposure. Moreover, we show that some jobs that were not known to be affected by previous waves of automation may now be subject to higher AI exposure. Finally, we find that some of the abilities where AI research is currently very intense are linked to tasks with comparatively limited labour input in the labour markets of advanced economies (e.g., visual and auditory processing using deep learning, and sensorimotor interaction through (deep) reinforcement learning). This article appears in the special track on AI and Society.
ZoPE: A Fast Optimizer for ReLU Networks with Low-Dimensional Inputs
Strong, Christopher A., Katz, Sydney M., Corso, Anthony L., Kochenderfer, Mykel J.
Deep neural networks often lack the safety and robustness guarantees needed to be deployed in safety critical systems. Formal verification techniques can be used to prove input-output safety properties of networks, but when properties are difficult to specify, we rely on the solution to various optimization problems. In this work, we present an algorithm called ZoPE that solves optimization problems over the output of feedforward ReLU networks with low-dimensional inputs. The algorithm eagerly splits the input space, bounding the objective using zonotope propagation at each step, and improves computational efficiency compared to existing mixed integer programming approaches. We demonstrate how to formulate and solve three types of optimization problems: (i) minimization of any convex function over the output space, (ii) minimization of a convex function over the output of two networks in series with an adversarial perturbation in the layer between them, and (iii) maximization of the difference in output between two networks. Using ZoPE, we observe a $25\times$ speedup on property 1 of the ACAS Xu neural network verification benchmark and an $85\times$ speedup on a set of linear optimization problems. We demonstrate the versatility of the optimizer in analyzing networks by projecting onto the range of a generative adversarial network and visualizing the differences between a compressed and uncompressed network.
PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training
Lee, Kimin, Smith, Laura, Abbeel, Pieter
Conveying complex objectives to reinforcement learning (RL) agents can often be difficult, involving meticulous design of reward functions that are sufficiently informative yet easy enough to provide. Human-in-the-loop RL methods allow practitioners to instead interactively teach agents through tailored feedback; however, such approaches have been challenging to scale since human feedback is very expensive. In this work, we aim to make this process more sample- and feedback-efficient. We present an off-policy, interactive RL algorithm that capitalizes on the strengths of both feedback and off-policy learning. Specifically, we learn a reward model by actively querying a teacher's preferences between two clips of behavior and use it to train an agent. To enable off-policy learning, we relabel all the agent's past experience when its reward model changes. We additionally show that pre-training our agents with unsupervised exploration substantially increases the mileage of its queries. We demonstrate that our approach is capable of learning tasks of higher complexity than previously considered by human-in-the-loop methods, including a variety of locomotion and robotic manipulation skills. We also show that our method is able to utilize real-time human feedback to effectively prevent reward exploitation and learn new behaviors that are difficult to specify with standard reward functions.