Goto

Collaborating Authors

 bell curve


Training Deep Neural Classifiers with Soft Diamond Regularizers

Adigun, Olaoluwa, Kosko, Bart

arXiv.org Machine Learning

We introduce new \emph{soft diamond} regularizers that both improve synaptic sparsity and maintain classification accuracy in deep neural networks. These parametrized regularizers outperform the state-of-the-art hard-diamond Laplacian regularizer of Lasso regression and classification. They use thick-tailed symmetric alpha-stable ($\mathcal{S \alpha S}$) bell-curve synaptic weight priors that are not Gaussian and so have thicker tails. The geometry of the diamond-shaped constraint set varies from a circle to a star depending on the tail thickness and dispersion of the prior probability density function. Training directly with these priors is computationally intensive because almost all $\mathcal{S \alpha S}$ probability densities lack a closed form. A precomputed look-up table removed this computational bottleneck. We tested the new soft diamond regularizers with deep neural classifiers on the three datasets CIFAR-10, CIFAR-100, and Caltech-256. The regularizers improved the accuracy of the classifiers. The improvements included $4.57\%$ on CIFAR-10, $4.27\%$ on CIFAR-100, and $6.69\%$ on Caltech-256. They also outperformed $L_2$ regularizers on all the test cases. Soft diamond regularizers also outperformed $L_1$ lasso or Laplace regularizers because they better increased sparsity while improving classification accuracy. Soft-diamond priors substantially improved accuracy on CIFAR-10 when combined with dropout, batch, or data-augmentation regularization.


Improving Uncertainty Sampling with Bell Curve Weight Function

Chong, Zan-Kai, Ohsaki, Hiroyuki, Goi, Bok-Min

arXiv.org Artificial Intelligence

Typically, a supervised learning model is trained using passive learning by randomly selecting unlabelled instances to annotate. This approach is effective for learning a model, but can be costly in cases where acquiring labelled instances is expensive. For example, it can be time-consuming to manually identify spam mails (labelled instances) from thousands of emails (unlabelled instances) flooding an inbox during initial data collection. Generally, we answer the above scenario with uncertainty sampling, an active learning method that improves the efficiency of supervised learning by using fewer labelled instances than passive learning. Given an unlabelled data pool, uncertainty sampling queries the labels of instances where the predicted probabilities, p, fall into the uncertainty region, i.e., $p \approx 0.5$. The newly acquired labels are then added to the existing labelled data pool to learn a new model. Nonetheless, the performance of uncertainty sampling is susceptible to the area of unpredictable responses (AUR) and the nature of the dataset. It is difficult to determine whether to use passive learning or uncertainty sampling without prior knowledge of a new dataset. To address this issue, we propose bell curve sampling, which employs a bell curve weight function to acquire new labels. With the bell curve centred at p=0.5, bell curve sampling selects instances whose predicted values are in the uncertainty area most of the time without neglecting the rest. Simulation results show that, most of the time bell curve sampling outperforms uncertainty sampling and passive learning in datasets of different natures and with AUR.


Crafting IT innovation strategies for real-world value

#artificialintelligence

Jeff Dirks is fascinated by new technologies like generative AI. But when it comes to implementation, the chief information and technology officer of workforce augmentation firm TrueBlue chooses a path that trails early adopters. "We're in the early majority," is the CIO/CTO's blunt self-assessment. Although many IT leaders would like to think of themselves -- and have others think of them -- as in the vanguard of new technology adoption, the vast majority find themselves in the middle of a bell curve, with innovators leading the way and laggards trailing behind, according to Everett Rogers' diffusion of innovations theory [see chart]. But there is no one "right" place to be along the curve.


Radial Basis Function Networks (RBFNs)

#artificialintelligence

In this article, we will talk about one of the algorithms that belong to the deep learning algorithms, RBFNs, as they are a special type of feeder neural network that use radial basis functions as activation functions. It has an input layer, a hidden layer, and an output layer and is mostly used for classification, regression, and time-series prediction. Radial basis function (RBF) networks are a common type of use in artificial neural networks for function approximation problems. Radial-based function networks are distinguished from other neural networks due to their global approximation and fast learning speed. The main advantage of the RBF network is that it has only one hidden layer and uses the radial basis function as the activation function.


How Eugenics Shaped Statistics - Issue 92: Frontiers

Nautilus

In early 2018, officials at University College London were shocked to learn that meetings organized by "race scientists" and neo-Nazis, called the London Conference on Intelligence, had been held at the college the previous four years. The existence of the conference was surprising, but the choice of location was not. UCL was an epicenter of the early 20th-century eugenics movement--a precursor to Nazi "racial hygiene" programs--due to its ties to Francis Galton, the father of eugenics, and his intellectual descendants and fellow eugenicists Karl Pearson and Ronald Fisher. In response to protests over the conference, UCL announced this June that it had stripped Galton's and Pearson's names from its buildings and classrooms. After similar outcries about eugenics, the Committee of Presidents of Statistical Societies renamed its annual Fisher Lecture, and the Society for the Study of Evolution did the same for its Fisher Prize. In science, these are the equivalents of toppling a Confederate statue and hurling it into the sea. Unlike tearing down monuments to white supremacy in the American South, purging statistics of the ghosts of its eugenicist past is not a straightforward proposition. In this version, it's as if Stonewall Jackson developed quantum physics. What we now understand as statistics comes largely from the work of Galton, Pearson, and Fisher, whose names appear in bread-and-butter terms like "Pearson correlation coefficient" and "Fisher information." In particular, the beleaguered concept of "statistical significance," for decades the measure of whether empirical research is publication-worthy, can be traced directly to the trio. Ideally, statisticians would like to divorce these tools from the lives and times of the people who created them. It would be convenient if statistics existed outside of history, but that's not the case.


Why Statistics is piece of a data science pie?

#artificialintelligence

As we all know, statistics is one of the industry knowledge one needs to be a data scientist. I want to start with my favorite line from the book "Rich dad poor dad " which is a non-fictional book about personal finance, investing, business, etc. Although that is not related to statistics, I relate that point with statistics. For me, the above words give the crystal clear explanation of what is statistics? As we all came across the phrase machine learning and deep learning are data-driven technologies.


Predictions on Performance Management Trends in 2019

#artificialintelligence

Performance management has been changing over these years from traditional annual reviews to periodic and continuous feedback. We, being a software tool provider, are able to experience the changes in customer (managers, HR and employees) expectations on their performance management software. Initially, it was mere, year-end review automation to categorize the employees into various ratings; then organizations started moving away from bell curves, ratings and showing interest on goals and achievement. Now more focus is shown on quarterly and monthly reviews before the annual appraisals are conducted. It is fact that highly engaged employees will perform better; feedback and developmental inputs will help employees work better and be more productive than mere rating them on their skills and categorizing.


The Dangers Of Our AI Models

#artificialintelligence

Picture a reality where whether you are at work or picking up your child from school, someone is tracking exactly where you are and logging it away for analysis. On the way home, you stop at the grocery store and pick up a six-pack and a few snacks. Someone is tracking those activities, too. Then when you bring those snacks to a party that night with your friends, all your conversations are captured and logged as well. The next day you have a meeting with a loan officer at a bank where you're hoping to get the money you need to start a business you've been working toward creating for years but are denied.


A Zero-Math Introduction to Markov Chain Monte Carlo Methods

#artificialintelligence

So, what are Markov chain Monte Carlo (MCMC) methods? In this article, I will explain that short answer, without any math. A parameter of interest is just some number that summarizes a phenomenon we're interested in. In general we use statistics to estimate parameters. For example, if we want to learn about the height of human adults, our parameter of interest might be average height in in inches.


A Zero-Math Introduction to Markov Chain Monte Carlo Methods

@machinelearnbot

So, what are Markov chain Monte Carlo (MCMC) methods? In this article, I will explain that short answer, without any math. A parameter of interest is just some number that summarizes a phenomenon we're interested in. In general we use statistics to estimate parameters. For example, if we want to learn about the height of human adults, our parameter of interest might be average height in in inches.