A few months ago, I wrote a post about optimization using gradient descent, which involves searching for a model that best meets certain criteria by repeatedly making adjustments that improve things a little bit at a time. In many situations, this works quite well and will always or almost always finds the best solution. But in other cases, it's possible for this approach to fall into a locally optimal solution that isn't the overall best, but is better than any nearby solution. A common way to deal with this sort of situation is to add some randomness into the algorithm, making it possible to jump out of one of these locally optimal solutions into a slightly worse solution that is adjacent to a much better one. In this post, I want to explore one such approach, called a genetic algorithm (or an evolutionary algorithm), which I'll illustrate with a specific type of genetic algorithm called symbolic regression.

Proceedings first published on the web in August, 2013. The original proceedings of the conference were distributed in hard-copy to attendees but were not published more widely. The proceedings published on the web comprise scanned copies of the original hard-copy papers. The scanning was performed by Tim Taylor in July 2013.

This paper presents a new model for word sense disambiguation formulated in terms of evolutionary game theory, where each word to be disambiguated is represented as a node on a graph whose edges represent word relations and senses are represented as classes. The words simultaneously update their class membership preferences according to the senses that neighboring words are likely to choose. We use distributional information to weigh the influence that each word has on the decisions of the others and semantic similarity information to measure the strength of compatibility among the choices. With this information we can formulate the word sense disambiguation problem as a constraint satisfaction problem and solve it using tools derived from game theory, maintaining the textual coherence. The model is based on two ideas: similar words should be assigned to similar classes and the meaning of a word does not depend on all the words in a text but just on some of them. The paper provides an in-depth motivation of the idea of modeling the word sense disambiguation problem in terms of game theory, which is illustrated by an example. The conclusion presents an extensive analysis on the combination of similarity measures to use in the framework and a comparison with state-of-the-art systems. The results show that our model outperforms state-of-the-art algorithms and can be applied to different tasks and in different scenarios.

Earlier this week, I found myself answering a question from a new colleague at Finning International that relates both to the research I do in the iSchool at the University of British Columbia, as well as the analytics, engineering & technology work that I lead at Finning. The questions were simple: 1) What is artificial intelligence? As I sat to reflect last evening, it dawned on me that taking time to craft a clear answer to these questions might be extremely beneficial for many. Analytics, data science, and predictive intelligence are hot topics in many communities and business areas. And yet, despite this interest, few folks I have talked to have a clear understanding of the history of the discipline; one, that frames much of the work currently going on within the space.

In this paper, we examine the problem of missing data in high-dimensional datasets by taking into consideration the Missing Completely at Random and Missing at Random mechanisms, as well as the Arbitrary missing pattern. Additionally, this paper employs a methodology based on Deep Learning and Swarm Intelligence algorithms in order to provide reliable estimates for missing data. The deep learning technique is used to extract features from the input data via an unsupervised learning approach by modeling the data distribution based on the input. This deep learning technique is then used as part of the objective function for the swarm intelligence technique in order to estimate the missing data after a supervised fine-tuning phase by minimizing an error function based on the interrelationship and correlation between features in the dataset. The investigated methodology in this paper therefore has longer running times, however, the promising potential outcomes justify the tradeoff. Also, basic knowledge of statistics is presumed.

I'm a programmer with no great mathematical background (2nd year university maths and stats, decades old and mostly forgotten) trying to teach myself about machine learning, and I found this book to be at exactly the right level for me. It's strongly oriented towards classifiers of one sort and another, and makes no claims to cover neural nets, genetic algorithms, genetic programming - but what it does cover it covers exceptionally clearly. I'd give it six stars out of five if it covered all aspects of machine learning, but I guess I can't have everything. In terms of writing style and comprehensibility this is probably one of the best textbooks I have ever read. I wish that it covered much much more, but what it does do it does remarkably well.

Data mining is everywhere, but its story starts many years before Moneyball and Edward Snowden. The following are major milestones and "firsts" in the history of data mining plus how it's evolved and blended with data science and big data. Data mining is the computational process of exploring and uncovering patterns in large data sets a.k.a. It is fundamental to data mining and probability, since it allows understanding of complex realities based on estimated probabilities. The goal of regression analysis is to estimate the relationships among variables, and the specific method they used in this case is the method of least squares.

I am currently building a genetic algorithm to tune n parameters where n will probably be in the range of 3 n 8 but could be up to 15. I would like my initial population N (let's say N 1000) to be evenly dispersed across the input space. When calculating the next generation I surmised that the most effective way to combine parents would be to calculate the centroid, on the surface of the hypersphere, between some m nearest-neighbour parents. The larger m is, the fewer new points we would add. The rest being calculated in a similar fashion but from random parents.

In the previous part of this series, I began discussing the field of advanced evolutionary artificial intelligence. AI has seen some stunning advancements in recent times, but we are still quite a while away from achieving the holy grail - general artificial intelligence. That is, an AI so developed that it could perform any cognitive task that a human can. To achieve this, we must look further than applying neural networks to specific tasks, we must look for algorithms that evolve and mutate to adapt to situations. What we're talking about are genetic algorithms; effectively the mathematical counterpart to Darwinian evolution.

"Mutation, it is the key to our evolution. It is how we have evolved from a single-celled organism into the dominant species on the planet. This process is slow, and normally taking thousands and thousands of years. Until few weeks back, it never occurred to me in so many years that above Darwinian quote from my all-time favourite sci-fi movie hints something about one of the most compelling theories in computer science I ever came across. Yes, I said – "Computer Science".