9.4.16 Common Statistical Tests for Comparing Data

Describes When to Use and Able to Interpret (But NOT Calculate) Hypothesis Tests Using: the Chi-Square Test,  Fisher’s Exact Test, Mcnemar’s Test, T-Test (Paired and Unpaired), ANOVA, ANCOVA, Wilcoxon Matched Pairs Signed Rank Test, Mann-Whitney U Test and Kruskal-Wallis Test

Hypothesis testing is an important statistical tool that is widely used in clinical research to determine whether a certain hypothesis is true or not. In order to apply a hypothesis test, one must first specify a null hypothesis, which represents the hypothesis that is being tested, and an alternative hypothesis, which represents the hypothesis that the researcher hopes to support.

There are several different types of hypothesis tests that can be used depending on the nature of the data being analyzed. Some of the most commonly used hypothesis tests in clinical research are:

Hypothesis Test	When to Use	Interpretation
Chi-Square Test	When comparing categorical data	Tests whether there is a significant association between two categorical variables
Fisher’s Exact Test	When sample sizes are small or expected frequencies are less than 5	Tests whether there is a significant association between two categorical variables
McNemar’s Test	When comparing paired data	Tests whether there is a significant difference between two paired proportions
T-Test (paired)	When comparing the means of two paired samples	Tests whether there is a significant difference between two means
T-Test (unpaired)	When comparing means of two independent samples	Tests whether there is a significant difference between two means
ANOVA	When comparing means of more than two independent samples	Tests whether there is a significant difference between means of multiple groups
ANCOVA	When comparing means of more than two independent samples, with a covariate	Tests whether there is a significant difference between means of multiple groups, controlling for a covariate
Wilcoxon Matched Pairs Signed Rank Test	When comparing medians of two paired samples	Tests whether there is a significant difference between two medians
Mann-Whitney U Test	When comparing medians of two independent samples	Tests whether there is a significant difference between two medians
Kruskal-Wallis Test	When comparing medians of more than two independent samples	Tests whether there is a significant difference between medians of multiple groups

Chi-square test:

The chi-square test is used to determine whether there is a significant association between two categorical variables. For example, it could be used to test whether there is a significant difference in the proportion of patients who respond to two different treatments.

Aspect	Details
When to Use Chi-square Test	Comparing observed frequencies with expected frequencies Categorical (nominal or ordinal) independent and dependent variables Testing for independence or goodness of fit in two or more categories Adequate sample size (expected frequency in each category should be at least 5)
Interpreting Results	Chi-square Value (χ²): Measures how much the observed frequencies differ from the expected frequencies P-Value: Indicates the significance of the results (less than 0.05 typically indicates significant difference) Degrees of Freedom: Calculated based on the number of categories Contingency Table: Used to display the frequencies and calculate the Chi-square statistic

Fisher’s exact test:

Fisher’s exact test is a variation of the chi-square test that is used when the sample size is small. It is used to determine whether there is a significant association between two categorical variables, similar to the chi-square test.

Aspect	Details
When to Use Fisher’s Exact Test	Used for categorical data in a 2×2 contingency table Ideal for small sample sizes Appropriate when expected frequencies in any of the cells are less than 5 Used to examine the significance of the association between two kinds of classification
Interpreting Results	P-Value: Measures the probability that the observed distribution of data would occur under the null hypothesis of no association between the variables Significance: If the p-value is less than a chosen significance level (usually 0.05), the null hypothesis of no association is rejected

McNemar’s test:

McNemar’s test is used to determine whether there is a significant difference in the proportion of patients who experience a certain outcome before and after treatment. For example, it could be used to test whether a new treatment is more effective at reducing symptoms of a disease compared to an existing treatment.

Aspect	Details
When to Use McNemar’s Test	Appropriate for paired nominal data Used to compare proportions in a 2×2 contingency table for matched pairs Ideal for “before and after” studies or studies with matched pairs Used to test for changes in responses due to an intervention or over time
Interpreting Results	Chi-square Value (or Test Statistic): Measures the difference in the paired proportions P-Value: Significance of the results (less than 0.05 typically indicates a significant difference) Assumption: The data pairs are dependent

T-test:

The t-test is used to determine whether there is a significant difference in the means of two normally distributed populations. There are two types of t-tests: paired t-tests, which are used when the samples are related, and unpaired t-tests, which are used when the samples are independent.

Aspect	Details
When to Use T-test	Comparing the means of two groups (Independent t-test for separate groups, Paired t-test for matched/paired data). Applicable when dealing with continuous data. Assumes normally distributed data, especially important for small sample sizes. Assumes homogeneity of variances when comparing two independent groups (equal variances).
Interpreting Results	T-Statistic: Measures the difference between the two group means relative to the spread or variability of their scores. P-Value: Indicates the significance of the results (less than 0.05 typically suggests a significant difference). Degrees of Freedom: Calculated based on the sample size(s). Confidence Intervals: This can provide additional insight into the range within which the true difference lies.

ANOVA:

Analysis of variance (ANOVA) is used to determine whether there is a significant difference in the means of three or more normally distributed populations. ANOVA can be used to compare the means of different treatments in a clinical trial.

Aspect	Details
When to Use ANOVA	Comparing means of 3+ groups One or more independent variables Continuous dependent variable Independent samples Normally distributed data Equal variances (Homoscedasticity)
Interpreting Results	F-Statistic: Ratio of variance between group means to variance within groups P-Value: Significance of the results (less than 0.05 indicates significant difference) Post Hoc Tests: Needed if ANOVA is significant to identify which groups differ Effect Size: Indicates the magnitude of the difference ANOVA Table: Includes degrees of freedom, sum of squares, mean squares, F-value, and p-value

ANCOVA:

Analysis of covariance (ANCOVA) is used to determine whether there is a significant difference in the means of three or more normally distributed populations while controlling for the effect of a covariate. For example, it could be used to test whether there is a significant difference in blood pressure between the two treatments while controlling for age.

Aspect	Details
When to Use ANCOVA	Comparing the means of two or more independent groups while controlling for one or more covariates (variables that could affect the outcome). Suitable for continuous dependent variables and categorical independent variables. Covariates are typically continuous variables. Used to enhance the statistical accuracy by reducing error variance.
Interpreting Results	Adjusted Means: ANCOVA adjusts the group means to what they would be if all groups had the same covariate values. F-Statistic and P-Value: Used to determine the significance of the independent variables and covariates on the dependent variable. Covariate Effect: Assess the impact and significance of the covariates included in the model. Assumptions: Includes homogeneity of regression slopes, along with assumptions similar to ANOVA (normality, independence, homoscedasticity).

Wilcoxon matched-pairs signed rank test:

The Wilcoxon matched-pairs signed rank test is a non-parametric test that is used to determine whether there is a significant difference between two related populations. It is often used when the data is not normally distributed.

Aspect	Details
When to Use Wilcoxon Matched-Pairs Signed-Rank Test	Appropriate for comparing two related samples or matched pairs. Used when the data is ordinal or the continuous data is not normally distributed. Ideal for “before and after” studies or studies with matched pairs where parametric test assumptions (like normality) are not met.
Interpreting Results	Test Statistic (W): The sum of signed ranks, which measures the difference between pairs. P-Value: Indicates the significance of the results (less than 0.05 typically indicates a significant difference). Assumptions: Assumes that the differences between pairs are symmetrically distributed around the median.

Mann-Whitney U test:

The Mann-Whitney U test is a non-parametric test that is used to determine whether there is a significant difference between two independent populations. It is often used when the data is not normally distributed.

Aspect	Details
When to Use Mann-Whitney U Test	Used for comparing two independent groups. Suitable when data is ordinal or continuous but not normally distributed. Ideal when the assumptions of the t-test are not met, especially for small sample sizes.
Interpreting Results	U Statistic: Measures the difference between the ranks of two independent samples. P-Value: Indicates the significance of the results (less than 0.05 typically indicates a significant difference). Assumptions: Assumes that the distributions of both groups are similar in shape.

Kruskal-Wallis test:

The Kruskal-Wallis test is a non-parametric test that is used to determine whether there is a significant difference between three or more independent populations. It is often used when the data is not normally distributed.

Aspect	Details
When to Use Kruskal-Wallis Test	Suitable for comparing the medians of three or more independent groups. Used when the data is ordinal or continuous but not normally distributed. Appropriate when the assumptions of ANOVA are not met, especially for small sample sizes or unequal variances.
Interpreting Results	H Statistic: The test statistic, similar to the F-statistic in ANOVA, measures the difference between groups. P-Value: Indicates the significance of the results (less than 0.05 typically indicates a significant difference). Assumptions: Assumes that the distributions of each group are similar.

In order to use these hypothesis tests effectively, it is important to understand the assumptions that underlie each test and to choose the appropriate test based on the nature of the data being analyzed. Additionally, it is important to interpret the results of the test appropriately and to consider the clinical significance of any differences that are found.

References:

1. Altman, D. G. (1991). Practical statistics for medical research. CRC press.
2. Armitage, P., Berry, G., & Matthews, J. N. (2002). Statistical methods in medical research. John Wiley & Sons.
3. Campbell, M. J., & Machin, D. (1999). Medical statistics: a commonsense approach