annoy
Chameleon2++: An Efficient Chameleon2 Clustering with Approximate Nearest Neighbors
Singh, Priyanshu, Ahuja, Kapil
Clustering algorithms are fundamental tools in data analysis, with hierarchical methods being particularly valuable for their flexibility. Chameleon is a widely used hierarchical clustering algorithm that excels at identifying high-quality clusters of arbitrary shapes, sizes, and densities. Chameleon2 is the most recent variant that has demonstrated significant improvements, but suffers from critical failings and there are certain improvements that can be made. The first failure we address is that the complexity of Chameleon2 is claimed to be $O(n^2)$, while we demonstrate that it is actually $O(n^2\log{n})$, with $n$ being the number of data points. Furthermore, we suggest improvements to Chameleon2 that ensure that the complexity remains $O(n^2)$ with minimal to no loss of performance. The second failing of Chameleon2 is that it lacks transparency and it does not provide the fine-tuned algorithm parameters used to obtain the claimed results. We meticulously provide all such parameter values to enhance replicability. The improvement which we make in Chameleon2 is that we replace the exact $k$-NN search with an approximate $k$-NN search. This further reduces the algorithmic complexity down to $O(n\log{n})$ without any performance loss. Here, we primarily configure three approximate nearest neighbor search algorithms (Annoy, FLANN and NMSLIB) to align with the overarching Chameleon2 clustering framework. Experimental evaluations on standard benchmark datasets demonstrate that the proposed Chameleon2++ algorithm is more efficient, robust, and computationally optimal.
How GitHub Copilot Simplified My Life as a Data Scientist
If you have been following the recent tech news, you might have heard about GitHub Copilot, an AI-based programming assistant. It's well and good if you are already using it, if not, keep reading! I have been using GitHub Copilot for a few months now, and I absolutely love it. In this article, I will try to make a convincing statement so that you give the Copilot a shot. You might fall in love too!
Machine-Learning Tips for Companies that Don't Want to Upset or Annoy Their Employees: Eye on A.I.
Although many companies talk about artificial intelligence, it's likely that the majority of their employees aren't actually using machine-learning technologies in the workplace. One big reason for that is while executives may be excited about A.I., employees may feel threatened or even insulted that managers would force them to use tools that that they fear will one day replace them. As FedEx senior data scientist Clayton Clouse said during an A.I. conference in San Francisco last week, "We shouldn't expect that people will jump up and down and be excited when we say, 'Hey, we're going to be augmenting your job with A.I.'" Citing a survey about A.I. from McKinsey, Clouse said that while the majority of companies polled by the consulting firm said they were implementing A.I. either in their business or through pilot projects, "only 6% reported that their employees were actually using the system the way they should be used." The employees, it turns out, are skeptical about A.I., especially machine-learning tools intended to automate decision-making in some way, Clouse said. If workers don't trust the A.I. tools to do as good of a job as them, they simply aren't going to use them, he explained.
New benchmarks for approximate nearest neighbors
One of my super nerdy interests include approximate algorithms for nearest neighbors in high-dimensional spaces. You have say 1M points in some high-dimensional space. Now given a query point, can you find the nearest points out of the 1M set? Doing this fast turns out to be tricky. I'm the author of Annoy which has more than 3,000 stars on Github.
AIB CIO Tim Hynes explains how AI can help CIOs now and in the future
Allied Irish Bank CIO Tim Hynes has seen many emerging technologies reach the hype cycle's peak during his IT career, now well into its third decade. Their history tells him that artificial intelligence will only reach its potential if it's promoted and applied to realistic scenarios. "Artificial intelligence feels a little bit like a gold rush event," said Hynes at the AI Congress London. "That doesn't mean there's not value in it. What it does mean is that if you're going to get real value, you have to be pragmatic."
spotify/annoy
Annoy (Approximate Nearest Neighbors Oh Yeah) is a C library with Python bindings to search for points in space that are close to a given query point. It also creates large read-only file-based data structures that are mmapped into memory so that many processes may share the same data. To install, simply do sudo pip install annoy to pull down the latest version from PyPI. For the C version, just clone the repo and #include "annoylib.h". There are some other libraries to do nearest neighbor search. Annoy is almost as fast as the fastest libraries, (see below), but there is actually another feature that really sets Annoy apart: it has the ability to use static files as indexes.
Amazon Echo's Alexa is getting push notifications, but they won't annoy you
Alexa, Amazon's digital assistant, has more than 12,000 skills available, and in the coming weeks some of them will be able to send notifications to your Echo and other Alexa-enabled devices. To start, just four skills will have notifications available including AccuWeather, JustEat, Life360, and The Washington Post. Shopping updates from Amazon's storefront are also coming presumably for item price tracking and the like. Amazon didn't say when notifications would go live for users other than it will happen "soon." These notifications won't work the same as they do on your phone, where every app immediately wants to start buzzing.