Unsupervised Clustering Algorithms

What is Clustering ?

Clustering is the process of assembling a set of observations under different groups based on some similarity criteria. Clustering is very essential in Machine learning since it helps to categorize our data set and helps us to understand our data well without the help of any sort of labeling of data. These algorithms do this by identifying underlying patterns in the feature set. However, different algorithms take different approach towards clustering the data points. The most common and widely used clustering algorithm is K-Means Clustering. But discussing about the algorithms in detail, let me introduce you to the method by which we measure the similarity between the observations

Similarity criteria

The similarity between the observations in the data set is generally done by distance metrics. There are three well-known and widely used distance metrics, namely, Manhattan, Euclidean and Minkowski distance.

If (x1, x2, …, xn) and (y1, y2, …, yn) are two given data points/ observations :

Manhattan Distance :

Euclidean Distance :

Minkowski Distance with order of norm = p :

The smaller is the value of a distance metric, the higher is the similarity between the two observations.

K-Means Clustering

The idea behind the algorithm is to group all the points to one of the K clusters with K cluster centroids. The entire clustering algorithm is based on the concept of Expectation-Maximization.

Steps involved in the algorithm :

However, the most important downside of this algorithm is that the algorithm is prone to find local optimum based on the starting points of cluster centroids. An effective solution to this issue is to repeat the algorithm for multiple times and take the results of that particular run which has the minimum Sum Squared Error (SSE). Sum squared error is defined as the sum of the distances of all the observations from their respective cluster centers.

One other down-side of this algorithm is that it tends to form globular clusters and are not suitable for grouping clusters with non-globular structures. To address these issues, we have other clustering algorithms known as hierarchical clustering algorithms.

Hierarchical Clustering

These clustering algorithms are based on the concept of hierarchy. It tries to find a hierarchical pattern in the unlabeled data. There are two kinds of hierarchical clustering :

What is Proximity Matrix ?

A matrix that stores the distances between each points. The distance can be based on any distance metric. If we have n observations in our data set, proximity matrix is of size n x n, with diagonal elements equal to 0.

Steps involved in Agglomerative Hierarchical clustering :

Types of Linkages :

What is a Dendrogram ?

A Dendrogram is a pictorial representation of the hierarchical tree formed as a result of hierarchical clustering. It aptly represents the clusters formed at different stages of the algorithm. It also gives a rough idea about the pint at which we need to terminate algorithm in order to get the right number of clusters.

Advantages of Hierarchical Clustering

However the biggest disadvantage of this algorithm is that, the performance of this algorithm degrades with increasing number of observations in the data set, since it becomes computationally expensive to calculate the huge n x n proximity matrix.

Related posts:

What Makes a Bad Relationship?

Everything might have started of well, but as time moved on you started to get more and more uneasy about the relationship. It does not necessarily mean that if you are going through a bad patch that… Read more

You Are Enough

How many times have you forced yourself into a shape that just didn’t feel quite like you? How many times have you forced something onto your body only for your body to reject it? Everyone seems to… Read more

300FIT meta And Fitness Influencers Participates In Content Creation!

The contactless home-training service 300FIT meta, signed a content creation contract with top-tier Korean fitness influencers. The 300FIT meta spokesperson announced that “To differentiate with… Read more