ML.NET - Get started in 10 minutes


Notepad), and save it as iris-data.txt When you paste the data it will look like the following. Each row represents a different sample of an iris flower. From left to right, the columns represent: sepal length, sepal width, petal length, petal width, and type of iris flower. If you're following along in Visual Studio, you'll need to configure iris-data.txt

Detecting Simpson’s Paradox

AAAI Conferences

Simpson’s paradox is the phenomenon that a trend of an association in the whole population reverses within the subpopulations defined by a categorical variable. Detecting Simpson’s paradox indicates surprising and interesting patterns of the data set for the user. It is generally discussed in terms of binary variables, but studies for the exploration of it for continuous variables are relatively rare. This paper describes a method to discover Simpson’s paradox for the trend of the pair of continuous variables. Correlation coefficient is used to indicate the association between a pair of continuous variables. We use categorical variables to partition the whole data set into groups. Our algorithm’s goal is to find the sign reversal between the coefficient correlations measured in the group relative to the original entire data. We show that our approach detects cases in real data sets as well as synthetic data sets, and demonstrate that our approach can uncover the hidden surprising pattern by detecting occurrences of Simpson’s paradox. This paper also proposes an approach that exploits sampled data for early Simpson’s paradox detection. We show the running time for the algorithm by examining through the combination of different conditions.

Clustering in Power BI using R


Here, I've used the famous Iris Flower dataset to show the clustering in Power BI using R. I've used the K-means clustering method to show the different species of Iris flower. About the dataset: The Iris dataset has 5 attributes (Sepal length, Sepal width, Petal width, Petal length, Species). The 3 different species are named as Setosa, Versicolor and Virginica. It is observed that, the Petal Length and Petal Width are similar in each Species, hence I have considered Petal Length for x axis and Petal Width for y axis to plot a graph. K-means Clustering: K means is a non-hierarchical iterative clustering technique.In this technique we start by randomly assigning the data points to clusters.



It is schema-based, and wraps scikit-learn. To get a feel for the library, consider the classic Iris dataset, where we predict the class of iris plant from measurements of the sepal, and petal. First, we create a schema describing our inputs and outputs. For our inputs, we have the length, and width, of both the sepal, and the petal. All of these input values happen to be numbers.

Feature engineering for building clustering models


We frequently get questions about whether we have chosen all the right parameters to build a machine learning model. There are two scenarios: either we have sufficient attributes (or variables) and we need to select the best ones OR we have only a handful of attributes and we need to know if these are impactful. Both are classic examples of feature engineering challenges. Most of the time, feature selection questions pop up as a prelude to model building. However, recently one of the trainees in our data science course had this question - based on his experience in working with some real data - "can we tell which attributes were most important in determining why a particular example (or a data point) ended up in a particular cluster?"