Multidimensional scaling

From CEOpedia
Multidimensional scaling
See also


Multidimensional scaling (MDS) - is a family of statistical techniques that take a table of similarities or dissimilarities between objects and turn it into a map: each object becomes a point in a space of few dimensions, placed so that the distances between points match the original dissimilarities as closely as possible[1]. Objects judged alike end up close together. Objects judged different end up far apart.

The input can be almost anything that expresses "how far apart" two things are. Consumers rating how similar two brands of beer feel. Travel times between cities. Correlations between test items. Co-purchase frequencies in a supermarket. The output is usually a two- or three-dimensional picture that a manager can read at a glance, which explains why MDS became a staple of marketing research, psychology and, later, data science. Unlike most statistical methods, MDS does not ask respondents to rate objects on attributes chosen in advance by the researcher. People simply say which things seem alike. The dimensions that organise those judgements are discovered afterwards.

History of multidimensional scaling

The mathematical roots go back to 1938, when Gale Young and Alston Householder showed how to recover point coordinates from a matrix of mutual distances. Psychology picked it up a decade later. Warren Torgerson published "Multidimensional scaling: I. Theory and method" in Psychometrika in 1952 and gave the field its name[2]. His procedure, now called classical or Torgerson scaling, treats the dissimilarities as actual distances measured on an interval or ratio scale, converts them into a matrix of scalar products and extracts coordinates with an eigen-decomposition. In 1966 the statistician John Gower showed that the same computation, which he called principal coordinates analysis, could be applied to many kinds of distance measures; that's why the method is sometimes labelled Torgerson-Gower scaling.

The big shift came in 1962. Roger Shepard, working at Bell Laboratories, argued that people's similarity judgements rarely behave like measured distances[3]. Someone can say that Pepsi and Coke are "more alike" than Coke and orange juice, but the numbers they attach to that feeling aren't trustworthy as ratios. Shepard's insight was that the rank order of the dissimilarities alone carries enough information to pin down a spatial configuration, provided there are enough objects. Only the order. Not the values.

Joseph Kruskal, also at Bell Labs, put the idea on firm footing two years later. His two 1964 papers in Psychometrika defined a badness-of-fit measure called stress and gave a numerical method (essentially steepest descent) for finding the configuration that minimises it[4][5]. The resulting approach became known as nonmetric MDS. Computer programs such as KYST, ALSCAL and, later, the SMACOF algorithm spread the technique through universities and market research agencies during the 1970s and 1980s.

Then the question of individual differences. Different consumers might share the same underlying dimensions but weigh them differently: one buyer cares mostly about price, another mostly about taste. J. Douglas Carroll and Jih-Jie Chang addressed this in 1970 with INDSCAL (individual differences scaling), which estimates one common space for the whole group plus a set of weights for each respondent[6].

Types of multidimensional scaling

MDS models differ in what they assume about the data and how many matrices they analyse at once:

  • Classical (Torgerson) MDS - treats dissimilarities as Euclidean distances and finds coordinates analytically through double centering and eigen-decomposition. No iterations. When the input really is a Euclidean distance matrix, the solution is exact and equivalent to principal component analysis of the underlying data.
  • Metric MDS (in the wider sense) - assumes the dissimilarities are linearly (or by some other known function) related to distances and minimises a loss function such as stress iteratively. Ratio, interval and spline transformations belong here.
  • Nonmetric (ordinal) MDS - assumes only that the order of the dissimilarities should be preserved. The algorithm fits a monotone regression of distances on dissimilarities, producing so-called disparities, and moves the points until the distances agree with those disparities. It suits survey judgements, rankings and other "soft" data.
  • Individual differences models (INDSCAL and relatives) - analyse several dissimilarity matrices, one per respondent or occasion, and produce a common group space together with individual dimension weights. A useful side effect: the INDSCAL axes are not rotationally arbitrary, so they can often be interpreted directly.
  • Unfolding - a variant for preference data, where respondents and objects are placed in the same space; each person is an "ideal point" and prefers objects that lie near it.
Type Data assumption Estimation Typical use
Classical Distances on ratio/interval scale Eigen-decomposition, one step Distance tables, starting configurations
Metric (stress-based) Linear or known function of distances Iterative minimisation Measured proximities with noise
Nonmetric Rank order only Monotone regression plus iteration Similarity ratings, rankings
INDSCAL Several matrices, shared dimensions Alternating least squares Segments, individual consumers

Methodology of classical MDS

Classical scaling follows a short, fixed sequence of steps[1][2].

Step 1. Build the dissimilarity matrix. Collect an n x n symmetric matrix \( D = [d_{ij}] \) with zeros on the diagonal. The entries may come from direct similarity ratings, sorting tasks, confusion data, or distances computed from attribute scores.

Step 2. Square the dissimilarities. Form \( D^{(2)} = [d_{ij}^2] \).

Step 3. Double centering. Using the centering matrix \( J = I - \frac{1}{n}\mathbf{1}\mathbf{1}^T \), compute

\[ B = -\frac{1}{2} J D^{(2)} J \]

Element by element this is

\[ b_{ij} = -\frac{1}{2}\left( d_{ij}^2 - \bar{d}_{i.}^2 - \bar{d}_{.j}^2 + \bar{d}_{..}^2 \right) \]

where the barred terms are the row mean, column mean and grand mean of the squared dissimilarities. The matrix B holds scalar products of the points measured from their centroid.

Step 4. Eigen-decomposition. Factor \( B = V \Lambda V^T \), with eigenvalues \( \lambda_1 \geq \lambda_2 \geq \dots \geq \lambda_n \) on the diagonal of \( \Lambda \).

Step 5. Coordinates. Keep the m largest positive eigenvalues (usually two or three) and compute

\[ X = V_m \Lambda_m^{1/2} \]

Each row of X gives one object's coordinates. The share of \( \lambda_1 + \dots + \lambda_m \) in the sum of the positive eigenvalues tells how much of the structure the reduced map keeps. If some eigenvalues come out negative, the dissimilarities can't be represented exactly in any Euclidean space, which is common with survey data.

Kruskal stress

Nonmetric and metric iterative methods judge a configuration by its stress. Kruskal's stress-1 is[4]:

\[ Stress_1 = \sqrt{\frac{\sum_{i<j}\left(d_{ij} - \hat{d}_{ij}\right)^2}{\sum_{i<j} d_{ij}^2}} \]

where \( d_{ij} \) are the distances between points in the map and \( \hat{d}_{ij} \) are the disparities, the values the distances should have to respect the dissimilarities (their rank order, in the nonmetric case). Zero means a perfect fit. The bigger the number, the more the map distorts the data.

Kruskal offered a verbal scale that is still quoted in almost every textbook:

Stress-1 Kruskal's verbal evaluation
0.20 Poor
0.10 Fair
0.05 Good
0.025 Excellent
0.00 Perfect

Use it with care. Stress tends to fall as dimensions are added and rise as more objects are scaled, so a value of 0.12 for 40 objects in two dimensions may be more impressive than 0.04 for eight objects in three. Borg and Groenen recommend looking also at the Shepard diagram (a scatterplot of distances against dissimilarities), the stability of the solution and, above all, whether the dimensions make substantive sense[1]. A common way to choose dimensionality is to plot stress against the number of dimensions and look for an "elbow".

Multidimensional scaling in marketing

Marketing adopted MDS early. By the late 1960s Paul Green and his colleagues were using it to draw perceptual maps: diagrams that show how customers see competing brands relative to each other[7]. The procedure runs roughly like this:

  • Choose 8-20 brands or products in a category.
  • Ask a sample of consumers to judge pairs of brands for similarity, or to sort brands into groups.
  • Average the judgements (or keep them separate for INDSCAL) and run MDS.
  • Interpret the axes, often by regressing attribute ratings on the coordinates, which projects attribute vectors such as "price" or "healthiness" into the map.
  • Look for empty regions in the space, check where the company's brand sits against the competition and decide whether to reposition.

Perceptual maps support market positioning, new product concept screening, advertising strategy and analysis of market structure. Two brands lying very close together are seen as substitutes, which is useful for pricing and for predicting cannibalisation. Combined with preference data through unfolding, the same map shows which market segments cluster around which brands, a link to customer segmentation and brand strategy.

Not everyone agreed it was the best tool for the job. Hauser and Koppelman compared perceptual maps built with factor analysis, discriminant analysis and similarity scaling, and concluded that factor analysis of attribute ratings did better on predictive ability, managerial interpretability and ease of use[8]. Similarity-based MDS kept its appeal where the researcher wants to avoid imposing attributes on respondents, or where the relevant attributes simply aren't known yet. Carroll and Green's 1997 review in the Journal of Marketing Research surveyed the main MDS models and their marketing applications three decades on[7].

Example of multidimensional scaling

Hypothetical example. A beverage company asks consumers to rate how different four bottled-water brands feel, on a scale from 0 (identical) to 10 (completely different). Average ratings:

A B C D
A 0 3 6 7
B 3 0 5 6
C 6 5 0 2
D 7 6 2 0

Squaring and centering. The squared dissimilarities have row means 23.5, 17.5, 16.25 and 22.25, and a grand mean of 19.875. For the first diagonal element: \( b_{11} = -\frac{1}{2}(0 - 23.5 - 23.5 + 19.875) = 13.5625 \). The full double-centred matrix is

\[ B = \begin{bmatrix} 13.5625 & 6.0625 & -8.0625 & -11.5625 \\ 6.0625 & 7.5625 & -5.5625 & -8.0625 \\ -8.0625 & -5.5625 & 6.3125 & 7.3125 \\ -11.5625 & -8.0625 & 7.3125 & 12.3125 \end{bmatrix} \]

Eigenvalues. The eigen-decomposition of B (computed in Python with NumPy) gives \( \lambda_1 = 34.43 \), \( \lambda_2 = 3.89 \), \( \lambda_3 = 1.43 \) and \( \lambda_4 = 0 \). Their sum, 39.75, equals the trace of B. None is negative, so these ratings happen to fit a Euclidean space exactly in three dimensions. The first dimension alone carries 86.6% of the total; the first two together carry 96.4%.

Coordinates. Multiplying the first two eigenvectors by \( \sqrt{\lambda_1} \) and \( \sqrt{\lambda_2} \):

Brand Dimension 1 Dimension 2
A -3.47 1.25
B -2.29 -1.52
C 2.34 0.07
D 3.42 0.20

(Signs of eigenvectors are arbitrary, so another program may return the map mirrored. Distances don't change.)

Reading the map. Distances in the two-dimensional map reproduce most pairs almost perfectly: A-B comes out at 3.00, A-C at 5.92, A-D at 6.97, B-C at 4.89 and B-D at 5.96. The weak spot is C-D, which the map shows as 1.09 instead of 2; the missing part of that difference lives in the third dimension. Treating the original ratings as the target values, stress-1 for the two-dimensional solution works out at about 0.074, between "good" and "fair" on Kruskal's scale.

What does it mean for the company? Dimension 1 splits the market cleanly: A and B on one side, C and D on the other. If attribute data showed that C and D are the premium mineral waters while A and B are budget spring waters, the first axis would read as a price/prestige dimension. Dimension 2 separates A from B, perhaps packaging or taste. C and D sit almost on top of each other, so consumers treat them as near substitutes. A new entrant wanting space of its own might aim for the empty middle of the map or the far end of dimension 2.

Advantages of multidimensional scaling

  • No predefined attributes needed. Respondents just compare objects. Hidden criteria that the researcher never thought of can still surface as dimensions.
  • Visual output. A two-dimensional map communicates competitive structure to managers faster than a table of numbers.
  • Works with weak data. Nonmetric MDS needs only rank order, which fits typical survey judgements better than methods assuming interval scales.
  • Flexible input. Any proximity measure works: ratings, sorting frequencies, co-occurrence counts, correlations, or distances computed from large behavioural datasets.
  • Individual differences. INDSCAL shows not only the shared structure but also how respondents or segments weigh each dimension.
  • Fast and deterministic in its classical form. Classical MDS is a single eigen-decomposition and often serves as the starting configuration for iterative methods.

Limitations of multidimensional scaling

  • Naming the dimensions is subjective. The algorithm produces axes, not labels. Interpretation needs extra attribute data or expert judgement, and two analysts may read the same map differently.
  • Rotation and reflection. Apart from INDSCAL, solutions can be rotated or mirrored freely, so axes have no fixed meaning.
  • Data collection grows quickly. Pairwise comparison of n objects needs n(n-1)/2 judgements: 45 pairs for 10 brands but 190 for 20, which tires respondents.
  • Local minima. Iterative nonmetric methods can stop at a poor configuration. Several random starts, or a classical start, are recommended.
  • Degenerate solutions. With few objects, nonmetric MDS can collapse points into clumps that give near-zero stress yet say nothing.
  • Averaging hides segments. Pooling all respondents into one matrix may produce a map that matches nobody's perception.
  • Fit is hard to judge. Stress benchmarks depend on the number of objects and dimensions, and Kruskal's scale was never meant as a strict test of statistical significance.
  • Measurement noise. Unreliable similarity ratings distort the map, the same issue discussed under measurement error.

Related methods

MDS belongs to the broader toolbox of multivariate data analysis and overlaps with several neighbours:

  • Principal component analysis starts from a matrix of variables rather than dissimilarities. Classical MDS on Euclidean distances gives the same configuration as PCA on the centred data.
  • Factor analysis (exploratory and confirmatory) explains correlations among observed variables with latent factors; in marketing it is the main attribute-based alternative to similarity-based perceptual maps.
  • Cluster analysis uses the same proximity matrices but produces discrete groups rather than continuous space. The two are often run together: clusters are drawn as circles on an MDS map.
  • Correspondence analysis maps the rows and columns of a contingency table (for example brands by attributes ticked by respondents) into a joint low-dimensional space and is widely used for brand image studies.
  • Semantic differential scales and other attribute ratings supply the data used to interpret MDS axes.

References

Footnotes

  1. ↑ Borg I., Groenen P.J.F., Modern Multidimensional Scaling: Theory and Applications, Springer, 2005
  2. ↑ Torgerson W.S., Multidimensional scaling: I. Theory and method, Psychometrika, 1952, pp. 401-419
  3. ↑ Shepard R.N., The analysis of proximities: Multidimensional scaling with an unknown distance function. I., Psychometrika, 1962, pp. 125-140
  4. ↑ Kruskal J.B., Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis, Psychometrika, 1964, pp. 1-27
  5. ↑ Kruskal J.B., Nonmetric multidimensional scaling: A numerical method, Psychometrika, 1964, pp. 115-129
  6. ↑ Carroll J.D., Chang J.J., Analysis of individual differences in multidimensional scaling, Psychometrika, 1970, pp. 283-319
  7. ↑ Carroll J.D., Green P.E., Psychometric methods in marketing research: Part II, Multidimensional scaling, Journal of Marketing Research, 1997, pp. 193-204
  8. ↑ Hauser J.R., Koppelman F.S., Alternative perceptual mapping techniques: Relative accuracy and usefulness, Journal of Marketing Research, 1979, pp. 495-506
Multidimensional scaling — recommended articles
Principal component analysis — Exploratory factor analysis — Cluster analysis — Multivariate data analysis — Market positioning — Perception of brand — Marketing research
Retrieved from CEOpedia