Goto

Collaborating Authors

 Memory-Based Learning


Memorization and Generalization in Neural Code Intelligence Models

arXiv.org Artificial Intelligence

Deep Neural Networks (DNNs) are increasingly being used in software engineering and code intelligence tasks. These are powerful tools that are capable of learning highly generalizable patterns from large datasets through millions of parameters. At the same time, their large capacity can render them prone to memorizing data points. Recent work suggests that the memorization risk manifests especially strongly when the training dataset is noisy, involving many ambiguous or questionable samples, and memorization is the only recourse. The goal of this paper is to evaluate and compare the extent of memorization and generalization in neural code intelligence models. It aims to provide insights on how memorization may impact the learning behavior of neural models in code intelligence systems. To observe the extent of memorization in models, we add random noise to the original training dataset and use various metrics to quantify the impact of noise on various aspects of training and testing. We evaluate several state-of-the-art neural code intelligence models and benchmarks based on Java, Python, and Ruby codebases. Our results highlight important risks: millions of trainable parameters allow the neural networks to memorize anything, including noisy data, and provide a false sense of generalization. We observed all models manifest some forms of memorization. This can be potentially troublesome in most code intelligence tasks where they rely on rather noise-prone and repetitive data sources, such as code from GitHub. To the best of our knowledge, we provide the first study to quantify memorization effects in the domain of software engineering and code intelligence systems. This work raises awareness and provides new insights into important issues of training neural models in code intelligence systems that are usually overlooked by software engineering researchers.


eGain Connects with IBM Watson Assistant for Smarter Service

#artificialintelligence

The connector leverages eGain's unique BYOB (Bring Your Own Bot) architecture, allowing business users to easily plug in the Watson Assistant into the eGain platform with no coding. Per Gartner, less than 10% of customer service journeys are fulfilled using self-service, which is why it is critical to integrate chatbots with human-assisted service channels such as live chat. The eGain Connector for Watson Assistant improves customer, agent, and business experiences at once. When customers escalate from Watson to human-assisted chat, their context is passed to the contact center agent so that they do not need to repeat information to the agent. Agents get to see interactions that customers have already had with Watson before they start their conversation with the customer.


Using tactile sensors and machine learning to improve how robots manipulate fabrics

#artificialintelligence

In recent years, roboticists have been trying to improve how robots interact with different objects found in real-world settings. While some of their efforts yielded promising results, the manipulation skills of most existing robotic systems still lag behinds those of humans. Fabrics are among the types of objects that have proved to be most challenging for robot to interact with. The main reasons for this are that pieces of cloth and other fabrics can be stretched, moved and folded in different ways, which can result in complex material dynamics and self-occlusions. Researchers at Carnegie Mellon University's Robotics Institute have recently proposed a new computational technique that could allow robots to better understand and handle fabrics.



5 Ways Companies Use Machine Learning to Improve Workplace Productivity

#artificialintelligence

Technology has become so advanced that, today, there's an app for almost anything, from children's education, to home improvement, to health monitoring, to workplace productivity. Gathering critical data to determine the best action to apply to specific situations has become integral in people's daily lives. Because of technology, critical decisions are now mostly based on scientific data. This makes every action more precise and error-free, especially in the business world. By using artificial intelligence and machine learning, industries can better cope with their consumers' demands.


Case-Based Reasoning (CBR) for the Self-Improving Help Desk

#artificialintelligence

In the AI-driven era, customer service has evolved to be more efficient and self-learning. AI systems help companies in a variety of ways including improving customer satisfaction ratings, reducing operational costs, and increasing revenue. AI has many other advantages for customer service that human agents cannot compete with -- it is always available, 24/7 and never gets tired or distracted. One of the leading AI systems in this area is CBR Systems' machine learning help desk system. Case Based Reasoning (CBR) is an AI technique that is increasingly used by customer service departments to improve their performance and help desk software providers to offer even more intelligent solutions for their customers.


5 Ways Machine Learning to Improve Your Digital Marketing

#artificialintelligence

One of the greatest things about digital marketing is that it is always at the forefront of the most recent online technologies. Machine learning is the most cutting-edge technology at the moment, and not just large companies have started to use it. Over 80% of online marketing agencies reported that their AI and machine-learning efforts had been deployed or increased since 2018, which is a long time ago. Machine learning is set to become the next step in harnessing data to take marketing efforts to new heights. These are five ways that machine learning can improve any marketing plan.


Kagan: IBM Watson Health Fails, Becomes Merative After Acquisition

#artificialintelligence

IBM Watson is a brilliant idea that failed in the marketplace. After being acquired and changing its name to Merative, will it win at the AI and IoT health game going forward?


Continual Variational Autoencoder Learning via Online Cooperative Memorization

arXiv.org Artificial Intelligence

Due to their inference, data representation and reconstruction properties, Variational Autoencoders (VAE) have been successfully used in continual learning classification tasks. However, their ability to generate images with specifications corresponding to the classes and databases learned during Continual Learning (CL) is not well understood and catastrophic forgetting remains a significant challenge. In this paper, we firstly analyze the forgetting behaviour of VAEs by developing a new theoretical framework that formulates CL as a dynamic optimal transport problem. This framework proves approximate bounds to the data likelihood without requiring the task information and explains how the prior knowledge is lost during the training process. We then propose a novel memory buffering approach, namely the Online Cooperative Memorization (OCM) framework, which consists of a Short-Term Memory (STM) that continually stores recent samples to provide future information for the model, and a Long-Term Memory (LTM) aiming to preserve a wide diversity of samples. The proposed OCM transfers certain samples from STM to LTM according to the information diversity selection criterion without requiring any supervised signals. The OCM framework is then combined with a dynamic VAE expansion mixture network for further enhancing its performance.


Generalization-Memorization Machines

#artificialintelligence

Firstly, we test the memorization ability and its influence of our HGMM on several small size datasets. The memory influence functions (i.e., formations (12), (13), (14) and (15)) were preloaded in our HGMM and evaluated by the m-fold cross validation (i.e., level-one-out validation, LOO for short). We set the baseline by setting the memory influence function be an identity matrix which is actually L2 loss SVM with decision (7) according to Theorem 4.3 (ii). Table II reports their highest LOO training and testing accuracies. From Table II, it is observed that our HGMM with either memory influence function has 100% training accuracies on all of these datasets.