McNemar and Mann-Whitney U Tests

Definition/Introduction

All well-designed research begins with a clearly defined question, often stated as a hypothesis. In clinical and translational research, the hypothesis often asks whether an intervention, diagnostic modality, biomarker, or clinical characteristic is associated with a measurable difference in outcome. Evaluating the hypothesis requires an appropriate study design, consistent and reproducible data collection, and statistical methods that align with the data structure. Therefore, statistical testing serves as an essential component of research. Statistical analysis helps determine whether observed differences likely reflect a true association or result from random variation.

Although statistical analysis historically required labor-intensive manual calculations, modern statistical software has made these analyses more accessible and efficient. Commonly used programs include SPSS (Statistical Package for the Social Sciences), Stata, SAS (Statistical Analysis System), R, and other statistical platforms. However, software cannot determine whether investigators have selected the correct test. Appropriate statistical testing requires an understanding of the research question, the outcome variable, the number of comparison groups, and whether the assumptions of a parametric test are met. Misclassification of the data structure or inappropriate test selection can lead to incorrect conclusions even when calculations are performed correctly.

Statistical tests are broadly classified as parametric or nonparametric. Parametric tests generally rely on several assumptions, including approximately normal distribution of continuous outcome data, independence of observations, and appropriate modeling of variance for the selected analysis. Nonparametric tests are appropriate when these assumptions are not met, sample sizes are small, data are ordinal, or outcome distributions are markedly skewed.[1][2]

McNemar Test

The McNemar test is a nonparametric statistical test designed for paired nominal data. The test is commonly used for binary outcomes when the same subjects are measured before and after an intervention or when 2 diagnostic methods are compared in the same individuals. A 2 × 2 contingency table is used to evaluate whether marginal proportions differ between paired observations. The analysis determines whether the proportion of subjects who change in one direction differs from the proportion who change in the opposite direction. Appropriate applications include paired yes–no, positive–negative, success–failure, and present–absent outcomes.

The McNemar test differs from the Fisher exact test and the chi-square test because it is designed for paired or matched binary observations. The Fisher exact test and the chi-square test generally analyze independent groups, whereas the McNemar test analyzes paired observations. For example, results from a diagnostic test obtained before and after an intervention in the same patient, or from 2 diagnostic tests performed on the same patient, constitute paired data. Statistical tests designed for independent samples ignore within-subject relationships and may yield inappropriate estimates of statistical significance.[3]

The McNemar test requires paired observations in which each subject contributes 2 binary measurements. Paired measurements may represent repeated observations over time or results from 2 diagnostic methods performed on the same subject. Most authors recommend at least 10 discordant pairs for reliable application of the asymptotic McNemar test. The formula for calculating the chi-square value for the McNemar test is as follows:

χ² = (b – c)² / (b + c)

In this equation, b represents the false-positive count and c represents the false-negative count. A significant chi-square value leads to rejection of the null hypothesis, indicating that the paired marginal proportions differ significantly. Whether the difference reflects improved diagnostic performance or treatment effectiveness depends on the direction of the discordant pairs and the clinical context.

The statistical power of the McNemar test is low when the sum of discordant pairs (b + c) is small (<25), even with a large total sample size. Therefore, research studies with fewer than 25 discordant pairs may use the exact binomial test. Alternatively, the Edwards continuity correction is another option.[4] The exact McNemar test is generally preferred for studies with few discordant pairs because the asymptotic approximation may be unreliable. Alternative approaches include the Edwards continuity correction and the McNemar mid-p test, which may provide greater statistical power while maintaining acceptable type I error rates.[5]

Mann-Whitney U test

The Mann-Whitney U test, also known as the Wilcoxon rank-sum test, is a nonparametric test used to compare ordinal or continuous outcomes between 2 independent groups. The test is frequently used when assumptions of an independent-samples t-test are not met, particularly when data are skewed, are measured on an ordinal scale, contain outliers, or are not normally distributed. Unlike the McNemar test, which analyzes paired binary data, the Mann-Whitney U test analyzes independent groups, such as outcomes between patients who received treatment A and patients who received treatment B.

The Mann-Whitney U test ranks all observations from both groups and compares the distribution of ranks rather than observed values directly. The test evaluates whether observations from 1 group tend to be higher or lower than observations from the other group. Since the test is rank-based rather than value-based, the Mann-Whitney U test is relatively robust to extreme outliers compared with the independent-samples t-test. This characteristic makes the test useful in biomedical research when continuous data are skewed, or sample sizes are too small to confidently assume normality. The Mann-Whitney U test is appropriate for comparing 2 independent groups on outcomes, such as pain scores, functional scores, time to healing, biomarker levels, hospital length of stay, or other continuous or ordinal variables when parametric assumptions are not satisfied.[6]

The Mann-Whitney U test is often interpreted as a comparison of group medians. However, this interpretation is valid only when distributions have similar shapes and differ primarily in location. The primary limitation of the Mann-Whitney U test is that it does not directly compare group means and should not be considered equivalent to a t-test. The test also assumes independence of observations between groups.

Therefore, the Mann-Whitney U test is inappropriate for paired measurements, repeated measurements from the same patient, matched case-control data, or before-and-after comparisons within the same subjects. Paired methods, such as the Wilcoxon signed-rank test, may be more appropriate in these situations. Reporting Mann-Whitney U test results should include group medians and interquartile ranges, the U statistic (or standardized z statistic when reported by statistical software), the P value, and a clear description of the independent groups compared. An appropriate measure of effect size should also be reported when possible.[7]

Clinical Pearls

Selection of a statistical test should be based on the structure of the data rather than the investigator’s preferred software or the appearance of the final P-value. Before selecting a test, the outcome should be identified as continuous, ordinal, nominal, or binary; groups should be classified as independent or paired; and assumptions of a parametric method should be assessed. The McNemar test is appropriate for paired binary outcomes, such as before-and-after responses in the same patients or results from 2 diagnostic tests applied to the same individuals. The analysis depends on discordant pairs rather than the total number of subjects or the number of concordant cells. A statistically significant result indicates a difference in marginal proportions but does not independently establish clinical superiority of one test or intervention.

The Mann-Whitney U test is appropriate for comparing ordinal or continuous outcomes between 2 independent groups when parametric assumptions are not met. The test compares rank distributions rather than group means. Interpretation as a comparison of medians is appropriate only when group distributions have similar shapes.

Statistical significance should always be interpreted together with effect size, confidence intervals, study design, sample size, and clinical relevance. A small P-value does not establish that an observed difference is clinically important, and a nonsignificant result does not necessarily establish the absence of a meaningful difference.[8][9][10][11][12]

Issues of Concern