K-Means Clustering is one of the most common unsupervised clustering algorithms, frequently used in data science, machine learning, and business intelligence for tasks such as customer segmentation and pattern recognition.

Core Principle

K-Means partitions data into $k$ distinct clusters based on their attributes. The algorithm works iteratively to assign each data point to one of $k$ clusters, aiming to minimize within-cluster variances.

Key Advantages

• Scalability: Suitable for large datasets.

• Generalization: Effective across different types of data.

• Performance: Can be relatively fast, depending on data and $k$ value choice. This makes it a go-to model, especially for initial exploratory analysis.

Limitations

• Dependence on Initial Seed: Results can vary based on the starting point, potentially leading to suboptimal solutions. Using multiple random starts or advanced methodologies like K-Means++ can mitigate this issue.

• Assumes Spherical Clusters: Works best for clusters that are somewhat spherical in nature. Clusters with different shapes or densities might not be accurately captured.

• Manual $k$ Selection: Determining the optimal number of clusters can be subjective and often requires domain expertise or auxiliary approaches like the elbow method.

• Sensitive to Outliers: Unusually large or small data points can distort cluster boundaries.

Measures of Variability

Within-cluster sum of squares (WCSS) evaluates how compact clusters are:

$$\text{WCSS} = \sum_{i=1}^{k} \sum_{x \in C_i} ||x - \mu_i||^2$$

Where:

• $k$ is the number of clusters.
• $C_i$ represents the $i$th cluster.
• $\mu_i$ is the mean of the $i$th cluster.

Evaluation Metrics

Silhouette Coefficient

The silhouette coefficient measures how well-separated clusters are. A higher value indicates better-defined clusters.

$$s(i) = \frac {b(i) - a(i)} { \max\{a(i), b(i)\}}$$

Where:

• $a(i)$: Mean intra-cluster distance for $i$ relative to its own cluster.
• $b(i)$: Mean nearest-cluster distance for $i$ relative to other clusters.

The silhouette score is then the mean of each data point’s silhouette coefficient.