Mann-Whitney U test

From CEOpedia
Mann-Whitney U test
See also


Mann-Whitney U test - is a nonparametric test that compares two independent samples by ranking all observations together and checking whether values from one group tend to be larger than values from the other[1]. It is also called the Wilcoxon rank-sum test or the Wilcoxon-Mann-Whitney (WMW) test. Researchers usually reach for it when the data are ordinal, when the sample is small, or when a normal distribution simply can't be assumed.

The idea is disarmingly simple. Forget the raw numbers; look only at their order. If two groups come from the same population, their ranks should be mixed more or less evenly. If one group keeps landing at the top of the ranking, something is going on. The test turns that intuition into a number, U, and tells you how surprising it would be under pure chance.

History of the Mann-Whitney U test

The method has two birth certificates. In 1945 Frank Wilcoxon, a chemist working for the American Cyanamid Company, published a four-page note in Biometrics Bulletin proposing that two treatments be compared by summing ranks instead of computing means[2]. He wanted something quick. Ranking a few laboratory results by hand was far faster than the arithmetic needed for a t statistic. Wilcoxon's note covered only equal group sizes, though.

Two years later Henry B. Mann, a mathematician at Ohio State University, and his graduate student Donald Ransom Whitney published "On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other" in the Annals of Mathematical Statistics (1947, vol. 18, no. 1, pp. 50-60)[3]. They defined the statistic U, allowed unequal sample sizes, built exact tables for samples up to eight observations each from a recurrence relation, and proved that U approaches a normal distribution as both samples grow. Mann and Whitney openly acknowledged that Wilcoxon had suggested an equivalent test. Which is why today's textbooks juggle three names for one procedure.

So: rank-sum (Wilcoxon) and U (Mann-Whitney) are two bookkeeping systems for the same information. Any statistics package that reports one can be converted into the other with a single line of algebra.

Assumptions

Nonparametric doesn't mean assumption-free. The Mann-Whitney test relies on the following conditions:

  • Independence of observations - within each group and between groups. Each customer, employee or firm appears once.
  • Two independent samples - the groups are not matched or paired. For before-and-after measurements on the same people a different test applies (see below).
  • At least ordinal measurement - the values must be meaningfully ordered. Data on an interval scale or ratio scale also qualify, but purely nominal categories (e.g. region names) do not.
  • Continuous underlying distribution - in theory, ties should not occur. In practice they do, and a correction handles them.
  • Similar shape of distributions - only needed if you want to interpret the result as a difference in medians. Without it, the test still compares the groups, but in a broader sense.

That last point trips up a lot of applied work. Fay and Proschan showed that the same WMW decision rule can be justified under several different sets of assumptions, and that what the p-value "means" depends on which set the analyst adopts[4]. If the two distributions have different spreads or different skew, a significant result does not automatically say the medians differ.

Hypotheses

Like every significance test, the Mann-Whitney procedure starts with a pair of statistical hypotheses.

The general (shape-free) version:

  • H0: the two populations are identical; a randomly chosen value from group 1 is equally likely to be larger or smaller than a randomly chosen value from group 2, i.e. \(P(X > Y) = 0.5\).
  • H1 (two-sided): \(P(X > Y) \neq 0.5\), one group tends to produce larger values.

The location-shift version (if both distributions have the same shape):

  • H0: median of group 1 = median of group 2.
  • H1: the medians differ (or, in a one-tailed test, one median is larger).

The one-sided form is used when the direction was stated before the data were collected. Choosing the direction after peeking at the numbers is cheating, plain and simple.

Formula of the Mann-Whitney U test

Let \(n_1\) and \(n_2\) be the sizes of the two samples and \(N = n_1 + n_2\). The procedure has five steps.

Step 1. Rank the pooled data. All N observations are sorted from smallest to largest and given ranks 1 to N. Tied values receive the average of the ranks they would otherwise occupy.

Step 2. Sum the ranks in each group. Call these \(R_1\) and \(R_2\). A useful check: \[R_1 + R_2 = \frac{N(N+1)}{2}\]

Step 3. Compute U for each group[5]: \[U_1 = n_1 n_2 + \frac{n_1(n_1+1)}{2} - R_1\] \[U_2 = n_1 n_2 + \frac{n_2(n_2+1)}{2} - R_2\] The two values always satisfy \(U_1 + U_2 = n_1 n_2\). Each one counts, across all \(n_1 n_2\) possible pairs of observations, how often a member of one group is outranked by a member of the other (a tie counts as one half).

Step 4. Take the smaller value as the test statistic: \[U = \min(U_1, U_2)\] For small samples U is compared with a critical value from published tables. The null hypothesis is rejected when U is less than or equal to the tabled value. That's the reverse of most tests, where large statistics are suspicious. Here a small U means one group almost never beats the other.

Step 5. Normal approximation for larger samples. When both samples exceed roughly 10 (some authors say 20), U is close to normally distributed with \[\mu_U = \frac{n_1 n_2}{2}\] \[\sigma_U = \sqrt{\frac{n_1 n_2 (N+1)}{12}}\] and the standardized statistic is \[z = \frac{U - \mu_U}{\sigma_U}\] Many authors subtract 0.5 from \(|U - \mu_U|\) as a continuity correction[6].

Tie correction

Ties shrink the true variance of U. If there are g groups of tied values and \(t_j\) is the number of observations in tie group j, the corrected standard deviation is \[\sigma_U = \sqrt{\frac{n_1 n_2}{12}\left[(N+1) - \frac{\sum_{j=1}^{g}(t_j^3 - t_j)}{N(N-1)}\right]}\] With a handful of ties the change is tiny. With Likert-type survey items, where dozens of respondents pick "4", it can matter a lot.

Effect size

A p-value says whether a difference is detectable; it says nothing about how big the difference is. Two measures are common.

The correlation-type effect size r divides z by the square root of the total sample size[7]: \[r = \frac{|z|}{\sqrt{N}}\] Cohen's rough benchmarks (0.1 small, 0.3 medium, 0.5 large) are often borrowed for it.

The common-language effect size (CL), popularised by McGraw and Wong, is the probability that a randomly picked observation from one group exceeds a randomly picked observation from the other[8]. For the Mann-Whitney test it falls straight out of U: \[CL = \frac{U_1}{n_1 n_2}\] (using whichever U corresponds to the direction of interest). A related figure, the rank-biserial correlation, equals \(r_{rb} = 1 - \frac{2U}{n_1 n_2}\). Managers usually find CL easier to grasp than any correlation: "there's a 79% chance that an old-process ticket takes longer than a new-process ticket" needs no statistics degree.

Example of the Mann-Whitney U test

A service desk manager pilots a new ticket-routing procedure. Resolution times (in hours) are recorded for a sample of tickets handled under the old procedure (group A, n1 = 7) and the new procedure (group B, n2 = 8). The data are skewed, as resolution times nearly always are, so a t-test looks risky.

  • Group A (old): 14, 18, 21, 25, 30, 33, 41
  • Group B (new): 9, 11, 12, 15, 18, 19, 22, 26

Ranking the pooled data (N = 15). The two values of 18 share ranks 6 and 7, so each gets 6.5.

Value (hours) Group Rank
9 B 1
11 B 2
12 B 3
14 A 4
15 B 5
18 A 6.5
18 B 6.5
19 B 8
21 A 9
22 B 10
25 A 11
26 B 12
30 A 13
33 A 14
41 A 15

Rank sums:

  • \(R_A = 4 + 6.5 + 9 + 11 + 13 + 14 + 15 = 72.5\)
  • \(R_B = 1 + 2 + 3 + 5 + 6.5 + 8 + 10 + 12 = 47.5\)
  • Check: \(72.5 + 47.5 = 120 = \frac{15 \cdot 16}{2}\). Correct.

U statistics: \[U_A = 7 \cdot 8 + \frac{7 \cdot 8}{2} - 72.5 = 56 + 28 - 72.5 = 11.5\] \[U_B = 7 \cdot 8 + \frac{8 \cdot 9}{2} - 47.5 = 56 + 36 - 47.5 = 44.5\] Check: 11.5 + 44.5 = 56 = n1 n2. So U = min(11.5; 44.5) = 11.5.

The value can be verified by direct counting. The old-process ticket of 14 hours is beaten (i.e. is shorter than) by 5 new-process tickets (15, 18, 19, 22, 26); the 18-hour ticket by 3 tickets plus half a point for the tie; 21 by 2; 25 by 1; 30, 33 and 41 by none. Total: 5 + 3.5 + 2 + 1 = 11.5.

Exact test. For n1 = 7 and n2 = 8 the critical value at α = 0.05 is 10 for a two-tailed test and 13 for a one-tailed test. Since 11.5 > 10, the two-tailed null hypothesis is not rejected. If the manager had stated in advance that the new procedure should be faster (one-tailed), 11.5 ≤ 13 and H0 would be rejected.

Normal approximation (shown for illustration; the samples are small): \[\mu_U = \frac{56}{2} = 28\] Without the tie correction: \(\sigma_U = \sqrt{\frac{56 \cdot 16}{12}} = \sqrt{74.67} = 8.641\).

With one tie group of size t = 2, \(t^3 - t = 6\) and \(N(N-1) = 210\): \[\sigma_U = \sqrt{\frac{56}{12}\left[16 - \frac{6}{210}\right]} = \sqrt{4.667 \cdot 15.971} = \sqrt{74.53} = 8.633\] \[z = \frac{11.5 - 28}{8.633} = -1.91\] The two-tailed p-value is about 0.056, the one-tailed about 0.028. With the continuity correction, z = -16/8.633 = -1.85 (two-tailed p ≈ 0.064). Same story as the exact table.

Effect size:

  • \(r = \frac{1.91}{\sqrt{15}} = 0.49\), a fairly large effect
  • \(CL = \frac{44.5}{56} = 0.79\): there's roughly a 79% chance that a randomly picked old-process ticket took longer than a randomly picked new-process ticket
  • Rank-biserial correlation: \(1 - \frac{2 \cdot 11.5}{56} = 0.59\)

The lesson is a practical one. The effect looks big, yet with fifteen tickets the two-tailed test falls just short of significance. A larger pilot would very likely settle the question; this is a textbook case of low statistical power rather than "no effect".

Interpretation

Reading a Mann-Whitney result well takes a bit more than checking p < 0.05.

  • Report the medians (and ideally interquartile ranges) of both groups next to U and p. Rank-based tests are naturally paired with rank-based measures of central tendency. In the example the medians are 25 hours (A) and 16.5 hours (B).
  • Say which direction the difference goes. U alone is symmetric and doesn't reveal it; the rank sums or medians do.
  • Be careful with the word "median". The test compares medians only if the distributions share the same shape. A box plot of each group is the fastest way to judge that.
  • Always add an effect size. With 5,000 survey respondents almost any difference becomes significant, and a CL of 0.52 is hardly worth a board presentation.
  • Non-significance is not proof of equality. It may reflect small sample size or high sampling error.

Use in business and management research

In quantitative research on organizations, much of the data is ordinal by nature. Rating scales, rankings and "how likely are you to recommend" questions all produce numbers whose distances aren't strictly equal. The Mann-Whitney test fits such data without pretending otherwise. Typical applications include:

  • comparing customer satisfaction scores between two stores, two service channels or two time periods,
  • testing whether employees in remote and office-based teams rate job autonomy differently,
  • comparing delivery lead times or defect counts for two suppliers, where a few extreme cases would wreck a mean-based test,
  • marketing research experiments, e.g. purchase-intention ratings for two package designs,
  • comparing financial ratios of two groups of firms (say, family-owned vs. publicly listed), since such ratios are often heavily skewed,
  • analysing pilot projects with small samples, like the service-desk example above.

The test is available in practically every package, from Excel add-ins and SPSS ("Independent-Samples Mann-Whitney U") to R (wilcox.test) and Python (scipy.stats.mannwhitneyu). Different programs label the output as U, W or the rank sum, so it's worth checking which convention is used before copying numbers into a report.

Advantages of the Mann-Whitney U test

  • Works with ordinal data, where means and standard deviations are of doubtful meaning.
  • Doesn't require normality. With skewed or asymmetrical distributions it keeps its nominal Type I error rate.
  • Resistant to outliers: an extreme value of 410 hours instead of 41 would leave the ranks, and therefore U, completely unchanged.
  • Exact p-values are available for small samples, which matters in pilot studies.
  • Efficiency is high. Even when the data really are normal, the test retains about 95% (3/π ≈ 0.955) of the t-test's asymptotic efficiency, and for heavy-tailed distributions it can be far more efficient than the t-test[6].
  • Easy to explain, especially through the common-language effect size.

Limitations of the Mann-Whitney U test

  • Information is discarded: ranking ignores how far apart values are. A 1-hour gap and a 30-hour gap count the same.
  • It tests something subtly different from the t-test (stochastic dominance, not means), which is often misreported.
  • Interpretation as a median test needs equal-shaped distributions; with unequal variances it can reject H0 even when the medians are identical[4].
  • Many ties (e.g. 5-point scales) reduce power and make exact tables inaccurate; the tie-corrected normal approximation or a permutation test is then preferable.
  • Only two groups at a time. Repeating it across many pairs inflates the Type I error rate unless a correction is applied.
  • It doesn't handle covariates. If firm size or region needs to be controlled for, a regression-type model (e.g. ordinal logistic regression) is needed.
  • Confidence intervals for the shift (the Hodges-Lehmann estimator) exist but aren't reported by default in many programs.

Comparison with related tests

The Mann-Whitney test belongs to a family of rank-based statistical methods. Choosing among them depends on the design.

Test Design Data What is compared
Independent-samples t-test 2 independent groups interval/ratio, approx. normal means
Mann-Whitney U test 2 independent groups ordinal or non-normal ranks, P(X > Y), medians if shapes are equal
Wilcoxon signed-rank test 2 related (paired) measurements ordinal or non-normal differences median of differences
Kruskal-Wallis test 3 or more independent groups ordinal or non-normal ranks across groups
One-way ANOVA 3 or more independent groups interval/ratio, normal, equal variances means

Versus the t-test. When the assumptions of the independent-samples t-test hold - normality and homogeneity of variance - the t-test is slightly more powerful and estimates the mean difference directly. When distributions are skewed or contain outliers, the Mann-Whitney test often wins on power, sometimes by a large margin. Some analysts run both; if they agree, fine; if they disagree, the data deserve a closer look rather than a choice of the "nicer" p-value.

Versus the Kruskal-Wallis test. Kruskal and Wallis extended the rank-sum idea to k groups in 1952[9]. It is the nonparametric counterpart of one-way analysis of variance. For exactly two groups the Kruskal-Wallis H statistic is equivalent to the squared z of the Mann-Whitney test (without continuity correction). After a significant Kruskal-Wallis result, pairwise Mann-Whitney tests with a Bonferroni or Holm adjustment are a common post hoc analysis.

Versus the Wilcoxon signed-rank test. Same inventor, different design. The signed-rank test is meant for paired data: the same employees before and after training, the same products rated by the same panel. Applying the Mann-Whitney test to paired data ignores the pairing and usually wastes power. Applying the signed-rank test to independent groups is simply wrong.

References

Footnotes

  1. ↑ Hollander M., Wolfe D.A., Chicken E., Nonparametric Statistical Methods, John Wiley & Sons, 2013
  2. ↑ Wilcoxon F., Individual Comparisons by Ranking Methods, Biometrics Bulletin, 1945, pp. 80-83
  3. ↑ Mann H.B., Whitney D.R., On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other, Annals of Mathematical Statistics, 1947, pp. 50-60
  4. ↑ Fay M.P., Proschan M.A., Wilcoxon-Mann-Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules, Statistics Surveys, 2010
  5. ↑ Siegel S., Castellan N.J., Nonparametric Statistics for the Behavioral Sciences, McGraw-Hill, 1988
  6. ↑ Conover W.J., Practical Nonparametric Statistics, John Wiley & Sons, 1999
  7. ↑ Fritz C.O., Morris P.E., Richler J.J., Effect size estimates: Current use, calculations, and interpretation, Journal of Experimental Psychology: General, 2012, pp. 2-18
  8. ↑ McGraw K.O., Wong S.P., A common language effect size statistic, Psychological Bulletin, 1992, pp. 361-365
  9. ↑ Kruskal W.H., Wallis W.A., Use of Ranks in One-Criterion Variance Analysis, Journal of the American Statistical Association, 1952, pp. 583-621
Mann-Whitney U test — recommended articles
Statistical_hypothesis — Statistical_significance — Analysis_of_variance — Normal_distribution — Statistical_power — One-tailed_test — Small_sample_size — Descriptive_statistics — Homogeneity_of_variance — Post_hoc_analysis
Retrieved from CEOpedia