Von Neumann's trace inequality

The Davies–Bouldin index (DBI) (introduced by David L. Davies and Donald W. Bouldin in 1979) is a metric for evaluating clustering algorithms. This is an internal evaluation scheme, where the validation of how well the clustering has been done is made using quantities and features inherent to the dataset. This has a drawback that a good value reported by this method does not imply the best information retrieval.

Preliminaries

Let C_i be a cluster of vectors. Let X_j be an n dimensional feature vector assigned to cluster C_i.

$S_{i}={\sqrt[{q}]{{\frac {1}{T_{i}}}\displaystyle \sum _{j=1}^{T_{i}}{|X_{j}-A_{i}|^{q}}}}$

Here $A_{i}$ is the centroid of C_i and T_i is the size of the cluster i. S_i is a measure of scatter within the cluster. Usually the value of q is 2, which makes this a Euclidean distance function between the centroid of the cluster, and the individual feature vectors. Many other distance metrics can be used, in the case of manifolds and higher dimensional data, where the euclidean distance may not be the best measure for determining the clusters. It is important to note that this distance metric has to match with the metric used in the clustering scheme itself for meaningful results.

$M_{i,j}=\left|\left|A_{i}-A_{j}\right|\right|_{p}={\sqrt[{p}]{\displaystyle \sum _{k=1}^{n}\left|a_{k,i}-a_{k,j}\right|^{p}}}$

$M_{i,j}$ is a measure of separation between cluster $C_{i}$ and cluster $C_{j}$.

$a_{k,i}$ is the kth element of $A_{i}$, and there are n such elements in A for it is an n dimensional centroid.

Here k indexes the features of the data, and this is essentially the Euclidean distance between the centers of clusters i and j when p equals 2.

Definition

Let R_i,j be a measure of how good the clustering scheme is. This measure, by definition has to account for M_i,j the separation between the i^th and the j^th cluster, which ideally has to be as large as possible, and S_i, the within cluster scatter for cluster i, which has to be as low as possible. Hence the Davies Bouldin Index is defined as the ratio of S_i and M_i,j such that these properties are conserved:

1. $R_{i,j}\geqslant 0$.
2. $R_{i,j}=R_{j,i}$.
3. if $S_{j}\geqslant S_{k}$ and $M_{i,j}=M_{i,k}$ then $R_{i,j}>R_{i,k}$.
4. and if $S_{j}=S_{k}$ and $M_{i,j}\leqslant M_{i,k}$ then $R_{i,j}>R_{i,k}$.

$R_{i,j}={\frac {S_{i}+S_{j}}{M_{i,j}}}$

This is the symmetry condition. Due to such a formulation, the lower the value, the better the separation of the clusters and the 'tightness' inside the clusters.

$D_{i}\equiv \max _{j:i\neq j}R_{i,j}$

If N is the number of clusters:

${DB}\equiv {\frac {1}{N}}\displaystyle \sum _{i=1}^{N}D_{i}$

DB is called the Davies Bouldin Index. This is dependent both on the data as well as the algorithm. D_i chooses the worst case scenario, and this value is equal to R_i,j for the most similar cluster to cluster i. There could be many variations to this formulation, like choosing the average of the cluster similarity, weighted average and so on.

Explanation

These conditions constrain the index so defined to be symmetric and non-negative. Due to the way it is defined, as a function of the ratio of the within cluster scatter, to the between cluster separation, a lower value will mean that the clustering is better. It happens to be the average similarity between each cluster and its most similar one, averaged over all the clusters, where the similarity is defined as S_i above. This affirms the idea that no cluster has to be similar to another, and hence the best clustering scheme essentially minimizes the Davies Bouldin Index. This index thus defined is an average over all the i clusters, and hence a good measure of deciding how many clusters actually exists in the data is to plot it against the number of clusters it is calculated over. The number i for which this value is the lowest is a good measure of the number of clusters the data could be ideally classified into. This has applications in deciding the value of k in the kmeans algorithm, where the value of k is not known apriori. The SOM toolbox contains a MATLAB implementation.

External links

• http://machaon.karanagai.com/validation_algorithms.html
• http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.17.2072
• http://books.google.com/books?id=HY8gB2OIqSoC

Notes and references

43 year old Petroleum Engineer Harry from Deep River, usually spends time with hobbies and interests like renting movies, property developers in singapore new condominium and vehicle racing. Constantly enjoys going to destinations like Camino Real de Tierra Adentro.