How Does K-Means Clustering Work?
Learn how K-Means clustering groups data by iterating assignment and centroid updates, how to choose K, and how to run it in scikit-learn with examples.
Expected Interview Answer
K-Means is an unsupervised clustering algorithm that partitions data into K groups by iteratively assigning each point to its nearest centroid and then moving each centroid to the mean of its assigned points until they stabilize.
You pick K in advance, initialize K centroids (often with k-means++), then repeat two steps: the assignment step labels every point by its closest centroid using squared Euclidean distance, and the update step recomputes each centroid as the mean of its members. This minimizes within-cluster sum of squares (inertia). It converges to a local optimum, so results depend on initialization; running multiple restarts and choosing K with the elbow or silhouette method is standard practice.
- Simple and fast, scaling near-linearly with the number of points
- Works well on large datasets and is easy to interpret
- Guarantees convergence to a local minimum
- Centroids give a compact, meaningful summary of each cluster
- Widely supported and easy to tune with a single main hyperparameter K
AI Mentor Explanation
Imagine spreading fielders across the outfield to cover where balls actually land. You place K fielders at guessed spots, then assign every past shot to its nearest fielder, and shift each fielder to the average landing point of the shots they cover. Repeat until nobody needs to move — the fielders now sit at the natural centres of the scoring zones, exactly like K-Means centroids settling over dense regions of data.
Step-by-Step Explanation
Step 1
Choose K
Decide how many clusters to find, using domain knowledge or the elbow/silhouette method.
Step 2
Initialize centroids
Place K centroids, preferably with k-means++ to spread them out and speed convergence.
Step 3
Assignment step
Assign each point to the nearest centroid using squared Euclidean distance.
Step 4
Update step
Move each centroid to the mean of all points currently assigned to it.
Step 5
Repeat until convergence
Loop assignment and update until assignments stop changing or inertia plateaus.
Step 6
Evaluate and restart
Run several random restarts and keep the solution with the lowest inertia.
What Interviewer Expects
- Clear description of the assign-then-update iterative loop
- Knowledge that it minimizes within-cluster sum of squares (inertia)
- Awareness that it finds a local, not global, optimum
- How to choose K (elbow, silhouette) and initialize (k-means++)
- Understanding of its assumptions: spherical, similarly sized clusters
Common Mistakes
- Claiming K-Means finds the global optimum every run
- Forgetting to scale/standardize features before clustering
- Assuming it works well on non-spherical or varied-density clusters
- Confusing K-Means with KNN (a supervised classifier)
- Not running multiple initializations to avoid poor local minima
Best Answer (HR Friendly)
“K-Means sorts data into a set number of groups by repeatedly assigning each item to its nearest group center and then recomputing each center from its members. It keeps adjusting until the groups stop changing, giving you natural clusters that summarize the data.”
Code Example
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import make_blobs
X, _ = make_blobs(n_samples=300, centers=4, random_state=42)
X = StandardScaler().fit_transform(X)
kmeans = KMeans(n_clusters=4, init='k-means++', n_init=10, random_state=42)
labels = kmeans.fit_predict(X)
print('Centroids:\n', kmeans.cluster_centers_)
print('Inertia:', kmeans.inertia_)inertias = []
for k in range(1, 10):
km = KMeans(n_clusters=k, n_init=10, random_state=42).fit(X)
inertias.append(km.inertia_)
# Plot k vs inertia and look for the 'elbow' where the drop flattens
for k, i in enumerate(inertias, start=1):
print(k, round(i, 2))Follow-up Questions
- How do you choose the right value of K?
- Why is feature scaling important before running K-Means?
- How does k-means++ initialization improve on random seeding?
- What are the limitations of K-Means on non-spherical clusters?
- How does K-Means differ from DBSCAN and hierarchical clustering?
MCQ Practice
1. What objective does standard K-Means minimize?
K-Means iteratively reduces the within-cluster sum of squared distances to centroids, also called inertia.
2. Why run K-Means with multiple initializations (n_init)?
K-Means converges only to a local optimum, so multiple restarts help find a lower-inertia solution.
3. Which is a key assumption of K-Means?
Because it uses Euclidean distance to a mean, K-Means works best on compact, roughly spherical, comparably sized clusters.
Flash Cards
What type of learning is K-Means? — Unsupervised clustering — it groups unlabeled data into K clusters.
What are the two repeating steps? — Assignment (point to nearest centroid) and update (centroid to mean of its points).
What quantity does K-Means minimize? — Within-cluster sum of squares, known as inertia.
Why use k-means++? — It spreads initial centroids apart, giving faster, more reliable convergence than random seeding.
How do you pick K? — Use the elbow method on inertia or the silhouette score.