Multidimensional Scaling (MDS)

Multidimensional Scaling (MDS)

Dimension reduction method

Proximity near when geometric distance short, far when long

Proximity matrix of (often Euclidean) distances $\boldsymbol{B}$ determined from feature matrix $\boldsymbol{X}$

Use singular value decomposition of $\boldsymbol{B}$ to obtain eigenvalues ranked largest to smallest

Largest eigenvalues “group” features into lower dimensional combinations

Largest eigenvalues “group” features into lower dimensional combinations

Adequacy of reduced $k$-dimensions criterion

$$P_k =  \frac{\sum_{i-1}^k \lambda_i}{\sum_{i-1}^{n-1} \lambda_i}$$

$P_k \ge 0.8$ suggest a reasonable fit

If negative or complex eigenvalues, $\boldsymbol{B}$ not PD

Adequacy suggested by Mardia, KV, Kent, JT, and Bibby, JM, (1979). Multivariate Analysis. London: Academic Press

$$P_k^(1) =  \frac{\sum_{i-1}^k \mid \lambda_i \mid}{\sum_{i-1}^{n-1} \mid \lambda_i \mid} \hskip 5mm \text{and} \hskip 5mm P_k^(2) =  \frac{\sum_{i-1}^k \lambda_i^2}{\sum_{i-1}^{n-1} \lambda_i^2}$$

If negative or complex eigenvalues, $\boldsymbol{B}$ not PD

Adequacy suggested by Sibson, R, (1979). Studies in the robustness of multidimensional scaling: Perturbational analysis of classical scaling. Journal of the Royal Statistical Society B, 41, 217-229.