Goto

Collaborating Authors

 Memory-Based Learning


Machine Learning Patentability in 2019: 5 Cases Analyzed and Lessons Learned Part 1

#artificialintelligence

Claims 1 and 8 as recited are not practically performed in the human mind. As discussed above, the claims recite monitoring operation of machines using neural networks, logic decision trees, confidence assessments, fuzzy logic, smart agent profiling, and case-based reasoning. . . .


Exploring the Memorization-Generalization Continuum in Deep Learning

arXiv.org Machine Learning

Human learners appreciate that some facts demand memorization whereas other facts support generalization. For example, English verbs have irregular cases that must be memorized (e.g., go->went) and regular cases that generalize well (e.g., kiss->kissed, miss->missed). Likewise, deep neural networks have the capacity to memorize rare or irregular forms but nonetheless generalize across instances that share common patterns or structures. We analyze how individual instances are treated by a model on the memorization-generalization continuum via a consistency score. The score is the expected accuracy of a particular architecture for a held-out instance on a training set of a fixed size sampled from the data distribution. We obtain empirical estimates of this score for individual instances in multiple datasets, and we show that the score identifies out-of-distribution and mislabeled examples at one end of the continuum and regular examples at the other end. We explore three proxies to the consistency score: kernel density estimation on input and hidden representations; and the time course of training, i.e., learning speed. In addition to helping to understand the memorization versus generalization dynamics during training, the C-score proxies have potential application for out-of-distribution detection, curriculum learning, and active data collection.


Machine Learning Patentability in 2019: 5 Cases Analyzed and Lessons Learned Part 1 JD Supra

#artificialintelligence

Claims 1 and 8 as recited are not practically performed in the human mind. As discussed above, the claims recite monitoring operation of machines using neural networks, logic decision trees, confidence assessments, fuzzy logic, smart agent profiling, and case-based reasoning. . . .


Why Overfitting is a Bad Idea and How to Avoid It (Part 1: Overfitting in general)

#artificialintelligence

We want our AI models to be as accurate as they can be. That's one of the selling points of AI -- that we can encode the best version of our past knowledge and have an automated model infer and apply our judgement. How can we tell when the model is accurate enough to trust? More importantly how can we tell if our efforts to improve accuracy are actually making the model worse? This situation can happen through a training problem called overfitting.


A Corrective View of Neural Networks: Representation, Memorization and Learning

arXiv.org Machine Learning

We develop a corrective mechanism for neural network approximation: the total available non-linear units are divided into multiple groups and the first group approximates the function under consideration, the second group approximates the error in approximation produced by the first group and corrects it, the third group approximates the error produced by the first and second groups together and so on. This technique yields several new representation and learning results for neural networks. First, we show that two-layer neural networks in the random features regime (RF) can memorize arbitrary labels for arbitrary points under under Euclidean distance separation condition using $\tilde{O}(n)$ ReLU or Step activation functions which is optimal in $n$ up to logarithmic factors. Next, we give a powerful representation result for two-layer neural networks with ReLU and smoothed ReLU units which can achieve a squared error of at most $\epsilon$ with $O(C(a,d)\epsilon^{-1/(a+1)})$ for $a \in \mathbb{N}\cup\{0\}$ when the function is smooth enough (roughly when it has $\Theta(ad)$ bounded derivatives). In certain cases $d$ can be replaced with effective dimension $q \ll d$. Previous results of this type implement Taylor series approximation using deep architectures. We also consider three-layer neural networks and show that the corrective mechanism yields faster representation rates for smooth radial functions. Lastly, we obtain the first $O(\mathrm{subpoly}(1/\epsilon))$ upper bound on the number of neurons required for a two layer network to learn low degree polynomials up to squared error $\epsilon$ via gradient descent. Even though deep networks can express these polynomials with $O(\mathrm{polylog}(1/\epsilon))$ neurons, the best learning bounds on this problem require $\mathrm{poly}(1/\epsilon)$ neurons.



IBM's Watson Center pitches AI for everyone, from chefs to engineers

#artificialintelligence

At the IBM Watson Experience Center, digital and physical worlds meet in a futuristic-looking lounge overlooking San Francisco's Financial District. "Regardless of the industry you're in, there's likely an application for AI … even as a chef," said IBM's data and AI engagement lead Euniq Nebo, as he stood before a 32-foot digital screen displaying human-size images of various professionals. A chef on the screen stepped forward and came to life. Nebo spoke of questions facing a restaurant chef, such as which cutting-edge tools to invest in, or whether to incorporate local produce into a cuisine. But IBM is betting its AI can "extract the insights" from data to help its clients stay ahead of the curve, Nebo said.


How to Build an Assistant Using IBM Watson (Part 1 of 2)

#artificialintelligence

In these two articles, I'll show you how to build an assistant that will control a mock smart home thermostat. These articles are intended to get you started with building assistants by creating something relevant in the real world. If you reach the end of the article and want to take a deeper dive or get help with a different chat framework such as Twilio, Drift, Lex, or something else, please leave me a comment. Below is the list of tools and services that we'll cover: I work in product management, and strictly speaking, a product manager doesn't need to possess tech skills to do their work. After all, that's what engineers are for.


IBM Watson: AI that's already transforming lives The MSP Hub

#artificialintelligence

It's time to put AI to work – real work. IBM Watson lives and works in the real world, putting'smart' into operation where it's really needed. To find out more about AI that knows your industry and works with tools you already use, watch the video.


Learning to See Analogies: A Connectionist Exploration

arXiv.org Artificial Intelligence

This dissertation explores the integration of learning and analogy-making through the development of a computer program, called Analogator, that learns to make analogies by example. By "seeing" many different analogy problems, along with possible solutions, Analogator gradually develops an ability to make new analogies. That is, it learns to make analogies by analogy. This approach stands in contrast to most existing research on analogy-making, in which typically the a priori existence of analogical mechanisms within a model is assumed. The present research extends standard connectionist methodologies by developing a specialized associative training procedure for a recurrent network architecture. The network is trained to divide input scenes (or situations) into appropriate figure and ground components. Seeing one scene in terms of a particular figure and ground provides the context for seeing another in an analogous fashion. After training, the model is able to make new analogies between novel situations. Analogator has much in common with lower-level perceptual models of categorization and recognition; it thus serves as a unifying framework encompassing both high-level analogical learning and low-level perception. This approach is compared and contrasted with other computational models of analogy-making. The model's training and generalization performance is examined, and limitations are discussed.