Skip to content

Clustering: Going Further

This page briefly introduces three clustering methods that go beyond hierarchical and k-means. They are not covered in the course but are useful to know about when the standard approaches fall short.

PAM (Partitioning Around Medoids)

PAM is a more robust alternative to k-means. Instead of using the mean of a cluster as its centre, PAM uses an actual data point (the medoid), which makes it less sensitive to outliers. It also accepts any distance matrix, not just Euclidean.

library(cluster)

pam_result <- pam(dist_matrix, k = 3)

# The medoids are real observations
pam_result$medoids

# Visualise
library(factoextra)
fviz_cluster(pam_result)

PAM is a good first alternative when k-means gives unstable results or when you are working with a non-Euclidean distance matrix such as Bray-Curtis.

DBSCAN (Density-Based Spatial Clustering)

DBSCAN finds clusters based on local density rather than distance to a centroid. It does not require you to specify k in advance and can find clusters of arbitrary shape. Points in low-density regions are labelled as noise (cluster 0) rather than forced into a cluster.

library(dbscan)

# Two parameters to tune:
# eps: neighbourhood radius
# minPts: minimum points to form a dense region
db <- dbscan(df_scaled, eps = 0.5, minPts = 5)

table(db$cluster)
# Cluster 0 = noise points

fviz_cluster(db, df_scaled, geom = "point")

The main challenge is choosing eps. A k-nearest-neighbour distance plot helps:

kNNdistplot(df_scaled, k = 5)
abline(h = 0.5, lty = 2)  # Look for the "knee"

DBSCAN is particularly useful for spatial data or whenever you expect irregularly shaped clusters or a meaningful noise class.

Fuzzy Clustering

Standard clustering assigns each observation to exactly one cluster (hard assignment). Fuzzy clustering instead gives each observation a membership probability for each cluster. This is useful when observations genuinely lie between groups rather than belonging clearly to one.

library(cluster)

fanny_result <- fanny(dist_matrix, k = 3)

# Membership matrix: rows sum to 1
head(fanny_result$membership)
#>      [,1]  [,2]  [,3]
#> [1,] 0.95  0.03  0.02  <- clearly in cluster 1
#> [2,] 0.38  0.34  0.28  <- genuinely ambiguous

Observations with high entropy across memberships (no dominant cluster) are worth examining closely — they may represent transitional states or outliers.

Which Method to Consider?

Situation Consider
Outliers distort k-means results PAM
Clusters have irregular shapes DBSCAN
Working with non-Euclidean distances PAM
Observations may belong to multiple groups Fuzzy clustering
No idea how many clusters exist DBSCAN

Further Reading