Practice: Applying K-Means to Simple Data

Okay, let's put theory into practice. You've learned about unsupervised learning, clustering, and the mechanics of the K-Means algorithm. Now, we'll walk through applying K-Means to a simple dataset to see it in action. The goal is to group data points without knowing their "true" labels beforehand, relying only on the positions of the points themselves.

We'll use a common approach in introductory examples: generating synthetic data. This is helpful because we can create data where we know there are distinct groups, making it easier to visually check if K-Means does a reasonable job finding them. We will use Python along with popular libraries like Scikit-learn for the K-Means algorithm and Plotly for visualization.

Setting Up Our Tools

First, ensure you have the necessary libraries. If you're working in an environment like Google Colab or Anaconda, these might already be installed. If not, you'd typically install them using pip:

pip install scikit-learn numpy plotly

For this example, we need numpy for numerical operations (especially creating our data), sklearn.cluster for the KMeans algorithm, and plotly.graph_objects for creating interactive plots suitable for the web.

Generating and Visualizing Simple Data

Let's create some 2-dimensional data that clearly falls into three groups or "blobs". Scikit-learn provides a handy function make_blobs for exactly this purpose.

import numpy as np
import plotly.graph_objects as go
from sklearn.datasets import make_blobs

# Generate synthetic data with 3 distinct clusters
X, _ = make_blobs(n_samples=150,  # Total number of points
                  centers=3,      # Number of clusters to generate
                  cluster_std=0.8,# Standard deviation of the clusters (spread)
                  random_state=42)# For reproducibility
                                  # We ignore the second output (y), which are the true labels

# X is now a NumPy array with 150 rows and 2 columns (our features)

# Let's visualize the raw data before clustering
fig_raw = go.Figure(data=[go.Scatter(
    x=X[:, 0],
    y=X[:, 1],
    mode='markers',
    marker=dict(color='#495057', size=7, opacity=0.8) # Use gray for raw data
)])

fig_raw.update_layout(
    title='Synthetic Data Points (Before Clustering)',
    xaxis_title='Feature 1',
    yaxis_title='Feature 2',
    width=600,
    height=450,
    plot_bgcolor='#f8f9fa' # Light background
)

# Display the plot (In a notebook/web environment)
# fig_raw.show()

Before applying K-Means, it's always a good idea to look at your data. Here’s the plot generated by the code above:

The synthetic data points plotted in 2D space. We can visually identify three potential groups.

As you can see, the points form three reasonably well-separated groups. Our eyes can perform this clustering task quite easily for this simple 2D data. Let's see if K-Means can replicate this.

Applying K-Means

Now we'll use Scikit-learn's KMeans implementation. We need to tell the algorithm how many clusters ($K$) to look for. Since we generated the data with 3 centers, let's set $K=3$.

from sklearn.cluster import KMeans

# Initialize the K-Means algorithm
# n_clusters is the most important parameter: the number of clusters (K)
# n_init='auto' uses an intelligent default for running the algorithm multiple times
# with different centroid seeds to improve results.
# random_state ensures reproducibility of the initialization.
kmeans = KMeans(n_clusters=3, n_init='auto', random_state=42)

# Fit the algorithm to the data X
# This is where K-Means iterates: assigning points to clusters and updating centroids.
kmeans.fit(X)

# After fitting, the model contains the results:
# 1. Cluster assignments for each data point:
cluster_labels = kmeans.labels_
# 2. Coordinates of the final cluster centers (centroids):
centroids = kmeans.cluster_centers_

# print("Cluster labels assigned to each point:", cluster_labels)
# print("Coordinates of final centroids:\n", centroids)

The fit() method runs the K-Means algorithm on our data X. The algorithm iteratively assigns each point to the nearest centroid and then recalculates the centroid positions based on the assigned points, until the centroids stabilize or a maximum number of iterations is reached.

The results are stored in the kmeans object:

• kmeans.labels_: An array where the $i$-th element indicates the cluster index (0, 1, or 2 in this case) assigned to the $i$-th data point in X.
• kmeans.cluster_centers_: A 2D array containing the final coordinates of the centroids for each cluster.

Visualizing the K-Means Results

Now, let's visualize the same data points, but this time color them according to the cluster labels assigned by K-Means. We'll also plot the final centroids found by the algorithm.

# Define colors for the clusters - using the suggested palette
cluster_colors = ['#4263eb', '#12b886', '#fd7e14'] # Indigo, Teal, Orange
centroid_color = '#f03e3e' # Red for centroids

# Create the plot
fig_clustered = go.Figure()

# Add data points, colored by cluster label
for i in range(3): # Loop through clusters 0, 1, 2
    points_in_cluster = X[cluster_labels == i]
    fig_clustered.add_trace(go.Scatter(
        x=points_in_cluster[:, 0],
        y=points_in_cluster[:, 1],
        mode='markers',
        marker=dict(color=cluster_colors[i], size=7, opacity=0.8),
        name=f'Cluster {i}'
    ))

# Add the centroids
fig_clustered.add_trace(go.Scatter(
    x=centroids[:, 0],
    y=centroids[:, 1],
    mode='markers',
    marker=dict(color=centroid_color, size=14, symbol='x', line=dict(width=3)),
    name='Centroids'
))

fig_clustered.update_layout(
    title=f'K-Means Clustering Results (K=3)',
    xaxis_title='Feature 1',
    yaxis_title='Feature 2',
    width=600,
    height=450,
    plot_bgcolor='#f8f9fa',
    legend_title_text='Legend'
)

# Display the plot
# fig_clustered.show()

Here's the resulting plot:

The same data points, now colored according to the cluster assigned by K-Means with $K=3$. The red 'x' markers indicate the final positions of the cluster centroids.

Interpretation

Compare the K-Means result plot with the initial plot of the raw data. You should see that K-Means has successfully identified the three distinct groups present in our synthetic data. Each color represents a cluster found by the algorithm, and the red 'x' marks show the center (mean position) of all points belonging to that cluster.

In this simple case, where the clusters are well-separated and roughly spherical, K-Means performs very well.

What if We Chose a Different K?

Remember the discussion about choosing $K$? Let's briefly consider what might happen if we instructed K-Means to find, say, $K=2$ clusters in this data. The algorithm would still run, but it would be forced to partition the three visible groups into only two clusters. Typically, it might merge two of the original groups or split one group across the two resulting clusters, depending on the initial centroid placement. Similarly, choosing $K=4$ would force the algorithm to split one or more of the natural groups into smaller, potentially less meaningful clusters. This exercise highlights that while K-Means is effective at partitioning data, the choice of $K$ significantly influences the outcome and its interpretation.

Summary

In this practice section, you've applied the K-Means algorithm to a simple, visually intuitive dataset. You saw how to:

1. Generate synthetic data suitable for clustering practice using make_blobs.
2. Visualize the raw data to understand its structure.
3. Initialize and fit the KMeans model from Scikit-learn, specifying the desired number of clusters ($K$).
4. Extract the cluster assignments (labels_) and centroid locations (cluster_centers_) from the fitted model.
5. Visualize the results by coloring data points according to their assigned cluster and plotting the final centroids.

This hands-on example demonstrates the core process of using K-Means for finding groups in unlabeled data. While real-world data is often more complex and higher-dimensional, the fundamental steps remain similar. You now have a practical foundation for understanding how K-Means works and how to implement it using common tools.