Skip to content

Clustering: Going Further

This page briefly introduces three clustering methods that go beyond hierarchical and k-means. They are not required for the course, but knowing what they do helps you recognise when hierarchical clustering or k-means may not be appropriate.

PAM (Partitioning Around Medoids)

PAM is a more robust alternative to k-means. Instead of using the mean of a cluster as its centre, PAM uses an actual observation as the centre of each cluster, which makes it more robust to outliers than k-means. It can also work with a dissimilarity matrix, so it is not restricted to Euclidean data.

library(cluster)
pam_result <- pam(dist_matrix, k = 3)
# The medoids are real observations
pam_result$medoids
# Visualise
library(factoextra)
# pam() on a dissimilarity matrix does not store the original
# data, so it must be supplied explicitly for the 2D layout
fviz_cluster(pam_result, data = df_scaled)

When to consider PAM

PAM is useful when outliers are influential or when your data are represented by a non-Euclidean dissimilarity such as Bray-Curtis.

DBSCAN (Density-Based Spatial Clustering)

DBSCAN finds clusters based on local density rather than distance to a centroid. It does not require you to specify k in advance and can find clusters of arbitrary shape. Points in low-density regions are labelled as noise (cluster 0) rather than forced into a cluster.

DBSCAN has two parameters to set: eps, the radius within which neighbouring points are considered part of the same dense region, and minPts, the minimum number of points required within that radius for a region to count as dense.

The right eps depends entirely on the scale and dimensionality of your data. There is no universal default: on the scaled USArrests data used elsewhere on this page, eps = 0.5 classifies every single observation as noise, which is why the k-nearest-neighbour distance plot below matters. Always read eps off that plot for your own dataset rather than reusing a fixed value.

This all-noise result is actually a useful illustration of a broader point: unlike k-means or a forced cutree() cut, which always partition data into exactly k groups whether or not real structure exists, DBSCAN can correctly report that no real clusters are present. See the Validation section of the main clustering page for more on this distinction.

library(dbscan)
db <- dbscan(df_scaled, eps = 1.15, minPts = 5)
table(db$cluster)
# Cluster 0 = noise points

# Visualise on the first two variables (or use PCA scores
# if you want a 2D summary of a higher-dimensional dataset)
plot(df_scaled[, 1:2],
     col = db$cluster,
     pch = 19,
     xlab = "Variable 1",
     ylab = "Variable 2",
     main = "DBSCAN clustering")

The main challenge is choosing eps. A k-nearest-neighbour distance plot helps:

kNNdistplot(df_scaled, k = 5)
abline(h = 1.15, lty = 2)  # Look for the "knee" (on USArrests, this sits well above 0.5)

When to consider DBSCAN

DBSCAN is particularly useful for spatial data or whenever you expect irregularly shaped clusters or a meaningful noise class.

Fuzzy Clustering

Standard clustering assigns each observation to exactly one cluster (hard assignment). Fuzzy clustering instead assigns each observation a degree of membership to each cluster. This is useful when observations genuinely lie between groups rather than belonging clearly to one.

library(cluster)
fanny_result <- fanny(dist_matrix, k = 3)
# Membership matrix: rows sum to 1
head(fanny_result$membership)
#>                 [,1]      [,2]      [,3]
#> Alabama    0.4676004 0.3144516 0.2179480
#> Alaska     0.4278809 0.3178707 0.2542484
#> Arizona    0.5092197 0.2945668 0.1962135  <- leans toward cluster 1, though not sharply
#> Arkansas   0.2934077 0.3787718 0.3278205  <- genuinely ambiguous across all three
#> California 0.4668527 0.3084149 0.2247324
#> Colorado   0.4542018 0.3236683 0.2221299

Observations with similar membership values across several clusters are worth examining closely. They may represent transitional states or observations that do not fit clearly into any group. Arkansas above is a good example: no cluster reaches even 0.4, so it sits meaningfully between groups rather than belonging clearly to one.

Which Method to Consider?

Situation Consider
Outliers strongly influence k-means PAM
Non-Euclidean dissimilarity PAM
Clusters have irregular shapes DBSCAN
A meaningful noise class is expected DBSCAN
Observations may lie between groups Fuzzy clustering

Further Reading