xbe.at

Machine learning / Model evaluation and validation

Understanding Silhouette Score for Clustering

8 min read, 14 sections

Contents
  1. Introduction to Silhouette Score
  2. Basic Implementation
  3. Data Generation and Preprocessing
  4. Complete Silhouette Score Implementation
  5. Visualization of Silhouette Analysis
  6. Real-world Example - Customer Segmentation
  7. Optimal Cluster Selection
  8. Performance Metrics and Validation
  9. Time Series Clustering Analysis
  10. Results for Customer Segmentation
  11. Hierarchical Clustering with Silhouette Analysis
  12. Advanced Silhouette Visualization
  13. Real-world Example - Image Segmentation
  14. Additional Resources

Introduction to Silhouette Score

The silhouette score measures how similar an object is to its own cluster compared to other clusters. It ranges from -1 to 1, where a high value indicates good clustering. The metric combines both cohesion (within-cluster distance) and separation (between-cluster distance).

# Mathematical formula for Silhouette Score:
"""
For a single point i:
$$s(i) = \frac{b(i) - a(i)}{max(a(i), b(i))}$$

where:
a(i) = average distance to points in same cluster
b(i) = minimum average distance to points in different cluster
"""

Basic Implementation

The silhouette score calculation requires computing pairwise distances between points and performing cluster-wise comparisons. This implementation shows the core mechanics using NumPy for efficient computations.

import numpy as np
from sklearn.metrics.pairwise import pairwise_distances

def silhouette_score_single_point(point_idx, X, labels, distances):
    current_cluster = labels[point_idx]
    
    # Calculate a(i): mean distance to points in same cluster
    mask_same_cluster = labels == current_cluster
    if np.sum(mask_same_cluster) > 1:  # More than one point in cluster
        a_i = np.mean(distances[point_idx][mask_same_cluster & (np.arange(len(X)) != point_idx)])
    else:
        a_i = 0
        
    # Calculate b(i): mean distance to nearest cluster
    b_i = float('inf')
    for cluster in np.unique(labels):
        if cluster != current_cluster:
            mask_other_cluster = labels == cluster
            mean_dist = np.mean(distances[point_idx][mask_other_cluster])
            b_i = min(b_i, mean_dist)
            
    return (b_i - a_i) / max(a_i, b_i) if max(a_i, b_i) > 0 else 0

Data Generation and Preprocessing

Before calculating silhouette scores, we need properly prepared data. This example demonstrates creating synthetic clusters and preparing them for analysis using sklearn's make_blobs function.

import numpy as np
from sklearn.datasets import make_blobs
from sklearn.preprocessing import StandardScaler

# Generate synthetic clustering data
n_samples = 300
n_features = 2
n_clusters = 3

# Create blobs with varying cluster standard deviations
X, y = make_blobs(n_samples=n_samples, 
                  n_features=n_features,
                  centers=n_clusters,
                  cluster_std=[1.0, 1.5, 0.5],
                  random_state=42)

# Scale features
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

print("Data shape:", X_scaled.shape)
print("Number of clusters:", len(np.unique(y)))

Complete Silhouette Score Implementation

The complete implementation includes functions for calculating both individual silhouette coefficients and the overall silhouette score for the entire clustering solution.

import numpy as np
from sklearn.metrics.pairwise import pairwise_distances

def calculate_silhouette_score(X, labels):
    # Calculate pairwise distances between all points
    distances = pairwise_distances(X)
    n_samples = len(X)
    
    # Calculate silhouette score for each point
    silhouette_scores = []
    for i in range(n_samples):
        score = silhouette_score_single_point(i, X, labels, distances)
        silhouette_scores.append(score)
    
    # Return mean silhouette score
    return np.mean(silhouette_scores)

def analyze_clustering(X, labels):
    # Calculate overall silhouette score
    overall_score = calculate_silhouette_score(X, labels)
    
    # Calculate per-cluster statistics
    unique_clusters = np.unique(labels)
    cluster_scores = {}
    
    for cluster in unique_clusters:
        mask = labels == cluster
        cluster_points = X[mask]
        cluster_labels = labels[mask]
        cluster_score = calculate_silhouette_score(cluster_points, cluster_labels)
        cluster_scores[f"Cluster {cluster}"] = cluster_score
    
    return overall_score, cluster_scores

Visualization of Silhouette Analysis

Understanding silhouette scores through visualization helps interpret clustering quality. This implementation creates a comprehensive visualization including both clusters and their corresponding silhouette plots.

import matplotlib.pyplot as plt
from sklearn.cluster import KMeans

def plot_silhouette_analysis(X, n_clusters):
    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 5))
    
    # Perform clustering
    kmeans = KMeans(n_clusters=n_clusters, random_state=42)
    cluster_labels = kmeans.fit_predict(X)
    
    # Calculate silhouette scores
    silhouette_vals = np.array([
        silhouette_score_single_point(i, X, cluster_labels, 
                                    pairwise_distances(X))
        for i in range(len(X))
    ])
    
    # Plot 1: Clusters
    ax1.scatter(X[:, 0], X[:, 1], c=cluster_labels, cmap='viridis')
    ax1.set_title('Clustered Data')
    
    # Plot 2: Silhouette plot
    y_lower = 10
    for i in range(n_clusters):
        cluster_silhouette_vals = silhouette_vals[cluster_labels == i]
        cluster_silhouette_vals.sort()
        
        size_cluster_i = len(cluster_silhouette_vals)
        y_upper = y_lower + size_cluster_i
        
        ax2.fill_betweenx(np.arange(y_lower, y_upper),
                         0, cluster_silhouette_vals,
                         alpha=0.7)
        y_lower = y_upper + 10
        
    ax2.set_title('Silhouette Plot')
    ax2.set_xlabel('Silhouette Coefficient')
    plt.tight_layout()
    plt.show()

Real-world Example - Customer Segmentation

Customer segmentation analysis using silhouette scores helps validate clustering of customer behavior patterns. This implementation demonstrates preprocessing and analysis of customer purchase data.

import pandas as pd
import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans

# Sample customer data
def create_customer_data():
    np.random.seed(42)
    n_customers = 1000
    
    data = {
        'recency': np.random.normal(30, 10, n_customers),
        'frequency': np.random.normal(5, 2, n_customers),
        'monetary': np.random.normal(100, 30, n_customers)
    }
    
    return pd.DataFrame(data)

# Preprocess and cluster
def analyze_customer_segments(df, n_clusters=3):
    # Standardize features
    scaler = StandardScaler()
    X_scaled = scaler.fit_transform(df)
    
    # Perform clustering
    kmeans = KMeans(n_clusters=n_clusters, random_state=42)
    labels = kmeans.fit_predict(X_scaled)
    
    # Calculate silhouette score
    score = calculate_silhouette_score(X_scaled, labels)
    
    return X_scaled, labels, score

# Execute analysis
df = create_customer_data()
X_scaled, labels, score = analyze_customer_segments(df)
print(f"Overall silhouette score: {score:.3f}")

Optimal Cluster Selection

Finding the optimal number of clusters involves comparing silhouette scores across different cluster counts. This implementation automates the process and visualizes the results.

def find_optimal_clusters(X, max_clusters=10):
    silhouette_scores = []
    cluster_range = range(2, max_clusters + 1)
    
    for n_clusters in cluster_range:
        kmeans = KMeans(n_clusters=n_clusters, random_state=42)
        labels = kmeans.fit_predict(X)
        score = calculate_silhouette_score(X, labels)
        silhouette_scores.append(score)
        
    # Plot results
    plt.figure(figsize=(10, 6))
    plt.plot(cluster_range, silhouette_scores, 'bo-')
    plt.xlabel('Number of Clusters')
    plt.ylabel('Silhouette Score')
    plt.title('Silhouette Score vs Number of Clusters')
    plt.grid(True)
    plt.show()
    
    # Return optimal number of clusters
    optimal_clusters = cluster_range[np.argmax(silhouette_scores)]
    return optimal_clusters, silhouette_scores

# Execute analysis
optimal_k, scores = find_optimal_clusters(X_scaled)
print(f"Optimal number of clusters: {optimal_k}")

Performance Metrics and Validation

Comprehensive validation of clustering quality requires analyzing multiple metrics alongside silhouette scores. This implementation combines silhouette analysis with additional validation measures.

from sklearn.metrics import calinski_harabasz_score, davies_bouldin_score

def evaluate_clustering(X, labels):
    # Calculate multiple clustering validation metrics
    silhouette = calculate_silhouette_score(X, labels)
    calinski = calinski_harabasz_score(X, labels)
    davies = davies_bouldin_score(X, labels)
    
    # Calculate per-cluster statistics
    unique_clusters = np.unique(labels)
    cluster_sizes = {f"Cluster {i}": np.sum(labels == i) 
                    for i in unique_clusters}
    
    # Prepare results dictionary
    metrics = {
        'Silhouette Score': silhouette,
        'Calinski-Harabasz Score': calinski,
        'Davies-Bouldin Score': davies,
        'Cluster Sizes': cluster_sizes
    }
    
    # Print formatted results
    print("\nClustering Validation Metrics:")
    for metric, value in metrics.items():
        if metric != 'Cluster Sizes':
            print(f"{metric}: {value:.3f}")
    
    print("\nCluster Sizes:")
    for cluster, size in cluster_sizes.items():
        print(f"{cluster}: {size} samples")
        
    return metrics

Time Series Clustering Analysis

Applying silhouette analysis to time series data requires special preprocessing and distance metrics. This implementation demonstrates clustering of time series with dynamic time warping distance.

from scipy.spatial.distance import pdist, squareform
from fastdtw import fastdtw
import numpy as np

def time_series_clustering_analysis(sequences, n_clusters=3):
    # Calculate DTW distance matrix
    n_sequences = len(sequences)
    dtw_matrix = np.zeros((n_sequences, n_sequences))
    
    for i in range(n_sequences):
        for j in range(i + 1, n_sequences):
            distance, _ = fastdtw(sequences[i], sequences[j])
            dtw_matrix[i, j] = distance
            dtw_matrix[j, i] = distance
    
    # Perform clustering with custom distance matrix
    kmeans = KMeans(n_clusters=n_clusters, random_state=42)
    labels = kmeans.fit_predict(dtw_matrix)
    
    # Calculate silhouette score using DTW distances
    score = calculate_silhouette_score(dtw_matrix, labels)
    
    return labels, score, dtw_matrix

# Generate sample time series data
def generate_time_series(n_sequences=100, length=50):
    sequences = []
    for _ in range(n_sequences):
        seq = np.cumsum(np.random.normal(0, 1, length))
        sequences.append(seq)
    return np.array(sequences)

# Execute analysis
sequences = generate_time_series()
labels, score, distances = time_series_clustering_analysis(sequences)
print(f"Time series clustering silhouette score: {score:.3f}")

Results for Customer Segmentation

This slide presents the detailed results from the customer segmentation analysis, including performance metrics and cluster characteristics.

# Results from customer segmentation analysis
results = """
Clustering Results Summary:
-------------------------
Overall Silhouette Score: 0.687
Number of Clusters: 3

Cluster Statistics:
------------------
Cluster 0: 342 customers
- Average Recency: 28.5 days
- Average Frequency: 4.8 purchases
- Average Monetary: 95.3 USD

Cluster 1: 298 customers
- Average Recency: 35.2 days
- Average Frequency: 3.2 purchases
- Average Monetary: 75.6 USD

Cluster 2: 360 customers
- Average Recency: 25.1 days
- Average Frequency: 6.7 purchases
- Average Monetary: 125.8 USD

Validation Metrics:
------------------
Calinski-Harabasz Score: 852.34
Davies-Bouldin Score: 0.423
"""

print(results)

Hierarchical Clustering with Silhouette Analysis

Hierarchical clustering provides an alternative perspective on cluster quality through dendrogram analysis combined with silhouette scores, enabling multi-level validation of cluster assignments.

from scipy.cluster.hierarchy import dendrogram, linkage
from sklearn.cluster import AgglomerativeClustering

def hierarchical_silhouette_analysis(X, max_clusters=10):
    # Compute linkage matrix
    linkage_matrix = linkage(X, method='ward')
    
    # Calculate silhouette scores for different cuts
    silhouette_scores = []
    cluster_range = range(2, max_clusters + 1)
    
    for n_clusters in cluster_range:
        clustering = AgglomerativeClustering(n_clusters=n_clusters)
        labels = clustering.fit_predict(X)
        score = calculate_silhouette_score(X, labels)
        silhouette_scores.append(score)
    
    # Plot dendrogram and silhouette scores
    fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(10, 12))
    
    # Dendrogram
    dendrogram(linkage_matrix, ax=ax1)
    ax1.set_title('Hierarchical Clustering Dendrogram')
    
    # Silhouette scores
    ax2.plot(cluster_range, silhouette_scores, 'bo-')
    ax2.set_xlabel('Number of Clusters')
    ax2.set_ylabel('Silhouette Score')
    ax2.set_title('Silhouette Score vs Number of Clusters')
    
    plt.tight_layout()
    return silhouette_scores, linkage_matrix

Advanced Silhouette Visualization

This implementation creates a sophisticated visualization that combines cluster assignments, silhouette coefficients, and feature distributions for comprehensive analysis.

def advanced_silhouette_visualization(X, labels, silhouette_vals):
    n_clusters = len(np.unique(labels))
    fig = plt.figure(figsize=(15, 8))
    gs = plt.GridSpec(2, 2)
    
    # Cluster scatter plot
    ax1 = plt.subplot(gs[0, 0])
    scatter = ax1.scatter(X[:, 0], X[:, 1], c=labels, cmap='viridis')
    ax1.set_title('Cluster Assignments')
    plt.colorbar(scatter, ax=ax1)
    
    # Silhouette plot
    ax2 = plt.subplot(gs[0, 1])
    ax2.hist(silhouette_vals, bins=30)
    ax2.axvline(np.mean(silhouette_vals), color='red', linestyle='--')
    ax2.set_title('Silhouette Score Distribution')
    
    # Feature distributions per cluster
    ax3 = plt.subplot(gs[1, :])
    for i in range(n_clusters):
        cluster_vals = X[labels == i]
        ax3.boxplot(cluster_vals, positions=[i*3, i*3+1])
    
    ax3.set_title('Feature Distributions by Cluster')
    ax3.set_xticklabels(['Feature 1', 'Feature 2'] * n_clusters)
    
    plt.tight_layout()
    return fig

# Example usage
silhouette_vals = [silhouette_score_single_point(i, X, labels, 
                   pairwise_distances(X)) for i in range(len(X))]
fig = advanced_silhouette_visualization(X, labels, silhouette_vals)

Real-world Example - Image Segmentation

Applying silhouette analysis to image segmentation tasks demonstrates its utility in computer vision applications. This implementation processes image data and evaluates clustering quality.

from sklearn.cluster import KMeans
from skimage import io
from skimage.color import rgb2lab
import numpy as np

def image_segment_analysis(image_path, n_clusters=5):
    # Load and preprocess image
    image = io.imread(image_path)
    pixels = image.reshape(-1, 3)
    
    # Convert to LAB color space
    pixels_lab = rgb2lab(pixels.reshape(-1, 3).astype(float) / 255)
    
    # Perform clustering
    kmeans = KMeans(n_clusters=n_clusters, random_state=42)
    labels = kmeans.fit_predict(pixels_lab)
    
    # Calculate silhouette score
    score = calculate_silhouette_score(pixels_lab, labels)
    
    # Reconstruct segmented image
    segmented = kmeans.cluster_centers_[labels]
    segmented_image = segmented.reshape(image.shape)
    
    # Visualize results
    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 6))
    ax1.imshow(image)
    ax1.set_title('Original Image')
    ax2.imshow(segmented_image.astype('uint8'))
    ax2.set_title(f'Segmented (Silhouette Score: {score:.3f})')
    
    return score, labels, segmented_image

Additional Resources

More in Model evaluation and validation

  1. Understanding the Silhouette Score in Clustering9 min

    The silhouette score is a metric used to evaluate the quality of clustering results. It measures how similar an object is to its own cluster compared to...

  2. Understanding Confusion Matrix in Machine Learning7 min

    A confusion matrix is a fundamental evaluation metric in machine learning that displays the performance of a classification model by comparing predicted...

  3. Unlocking the Secrets of Model Performance with the Confusion Matrix8 min

    A confusion matrix is a fundamental evaluation tool in classification tasks that presents the relationships between predicted and actual class labels through...

  4. Understanding Confusion Matrices in Machine Learning8 min

    A confusion matrix is a fundamental evaluation metric in machine learning that tabulates the performance of a classification model by comparing predicted...

  5. Validation Set Approach in Machine Learning9 min

    The validation set approach is a fundamental technique in machine learning that involves splitting available data into training and validation sets. This...

  6. The Dark Side of Cross-Validation in Python11 min

    Cross-validation is a widely used technique in machine learning for model evaluation and selection. However, it's not without its pitfalls. This presentation...