Collaborating Authors

optimization algorithm

Hurwitz-Riemann Zeta And Other Special Probability Distributions - AI Summary


All the solutions were probability distributions, and in this article we introduce an even larger, generic class of problems (chaotic discrete dynamical systems) with known solution. Each dynamical system discussed here (or in my previous article) comes with two distributions: The name Hurwitz and Riemann-Zeta is just a reminder of their strong connection to number theory problems such as continued fractions, approximation of irrational numbers by rational ones, the construction and distribution of the digits of random numbers in various numeration systems, and the famous Riemann Hypothesis that has a one million dollar prize attached to it. The most well known probability distribution related to these functions is the discrete Zipf distribution. The author defines a family of distribution that generalizes the exponential power, normal, gamma, Weibull, Rayleigh, Maxwell-Boltzmann and chi-squared distributions, with applications in actuarial sciences. Our Hurwitz-Riemann Zeta distribution is yet another example arising this time from discrete dynamical systems, continuous on [0, 1].

6 Awesome Python Libraries for Data Science for 2022


This blog covers the 6 famous Python libraries for data science that are easy to use, have extensive documentation, and can perform computations faster. Data scientist is the sexiest job of the 21st century, but what is a data scientist without data? Harvard Business Review labels data as the new oil. There is a massive dearth of people qualified for data-related jobs. As a beginner, you can be tempted to wet your feet in the ever-evolving field of data science.

C++ Machine Learning Algorithms Inspired by Nature


This online course is for students and software developers who want to level up their skills by learning interesting optimization algorithms in C . You will learn some of the most famous AI algorithms by writing it in C from scratch, so we will not use any libraries. We will start with the Genetic Algorithm (GA), continue with Simulated Annealing (SA) and then touch on a less known one: Differential Evolution. Finally, we will look at Ant Colony Optimization (ACO). The Genetic Algorithm is the most famous one in a class called metaheuristics or optimization algorithms. You will learn what optimization algorithms are, when to use them, and then you will solve two problems with the Genetic Algorithm(GA).

Multiobjective Tree-Structured Parzen Estimator

Journal of Artificial Intelligence Research

Practitioners often encounter challenging real-world problems that involve a simultaneous optimization of multiple objectives in a complex search space. To address these problems, we propose a practical multiobjective Bayesian optimization algorithm. It is an extension of the widely used Tree-structured Parzen Estimator (TPE) algorithm, called Multiobjective Tree-structured Parzen Estimator (MOTPE). We demonstrate that MOTPE approximates the Pareto fronts of a variety of benchmark problems and a convolutional neural network design problem better than existing methods through the numerical results. We also investigate how the configuration of MOTPE affects the behavior and the performance of the method and the effectiveness of asynchronous parallelization of the method based on the empirical results.

Optimizer in Deep Learning


An optimizer is a function or an algorithm that customizes the attributes of the neural network, such as weights and discovering rate. Hence, it assists in decreasing the overall loss and also enhance the accuracy. The problem of picking the ideal weights for the version is an overwhelming job, as a deep learning version usually includes numerous parameters. It increases the requirement to pick an appropriate optimization algorithm for your application. You can utilize different optimizers to make changes in your weights as well as learning price.

Optimizers in Machine Learning


The optimizer is a crucial element in the learning process of the ML model. PyTorch itself has 13 optimizers, making it challenging and overwhelming to pick the right one for the problem. In this tutorial, I will go through the five most popular optimizers explaining their strengths and limits along with the math behind them. So, let's get into it! The ultimate goal of ML model is to reach the minimum of the loss function.

Exploding Gradients in Neural Networks


Get exclusive access to writing opportunities and advice in our community Discord. Exploding Gradients in Neural Networks is the way and scale calculated during the training of a neural network. It is used to keep informed of the network weights in the right path and by the right amount. Exploding Gradients may collect during an update and outcome in very big gradients in deep networks or recurrent neural networks. The standards of weights may develop as bulky as to overflow and result in NaN values at a risky.

The unsupervised reinforcement learning benchmark


Reinforcement Learning (RL) is a powerful paradigm for solving many problems of interest in AI, such as controlling autonomous vehicles, digital assistants, and resource allocation to name a few. We've seen over the last five years that, when provided with an extrinsic reward function, RL agents can master very complex tasks like playing Go, Starcraft, and dextrous robotic manipulation. While large-scale RL agents can achieve stunning results, even the best RL agents today are narrow. Most RL algorithms today can only solve the single task they were trained on and do not exhibit cross-task or cross-domain generalization capabilities. A side-effect of the narrowness of today's RL systems is that today's RL agents are also very data inefficient.

A Brief Overview of Physics-inspired Metaheuristic Optimization Techniques Artificial Intelligence

Metaheuristic algorithms are methods devised to efficiently solve computationally challenging optimization problems. Researchers have taken inspiration from various natural and physical processes alike to formulate meta-heuristics that have successfully provided near-optimal or optimal solutions to several engineering tasks. This chapter focuses on meta-heuristic algorithms modelled upon non-linear physical phenomena having a concrete optimization paradigm, having shown formidable exploration and exploitation abilities for such optimization problems. Specifically, this chapter focuses on several popular physics-based metaheuristics as well as describing the underlying unique physical processes associated with each algorithm.

A new Sparse Auto-encoder based Framework using Grey Wolf Optimizer for Data Classification Problem Artificial Intelligence

One of the most important properties of deep auto-encoders (DAEs) is their capability to extract high level features from row data. Hence, especially recently, the autoencoders are preferred to be used in various classification problems such as image and voice recognition, computer security, medical data analysis, etc. Despite, its popularity and high performance, the training phase of autoencoders is still a challenging task, involving to select best parameters that let the model to approach optimal results. Different training approaches are applied to train sparse autoencoders. Previous studies and preliminary experiments reveal that those approaches may present remarkable results in same problems but also disappointing results can be obtained in other complex problems. Metaheuristic algorithms have emerged over the last two decades and are becoming an essential part of contemporary optimization techniques. Gray wolf optimization (GWO) is one of the current of those algorithms and is applied to train sparse auto-encoders for this study. This model is validated by employing several popular Gene expression databases. Results are compared with previous state-of-the art methods studied with the same data sets and also are compared with other popular metaheuristic algorithms, namely, Genetic Algorithms (GA), Particle Swarm Optimization (PSO) and Artificial Bee Colony (ABC). Results reveal that the performance of the trained model using GWO outperforms on both conventional models and models trained with most popular metaheuristic algorithms.