Significance tests for two dependent samples are a study of correlated samples. This includes the before-after effect and matched paired study. McNemar’s, Marginal Homogeneity, Sign and the Wilcoxon test are non parametric tests that are used for two dependent samples. The McNemar test is the best test for dichotomous variables with two dependent sample studies. When a category of the sample is more, then the two marginal homogeneity tests are the best tests for dependent samples. When the dependent variable samples are continuous in nature, then the sign and Wilcoxon tests are the best tests for two dependent sample studies.
Key terms and concepts
- Dependent sample: Two correlated samples, or a before and after study of a sample, are examples of the dependent sample. There are two types of dependency: (1) before after sample with the same people at different points of time and (2) matched paired study with the same people at different points of time.
- Type of significance estimate: Asymptotic, exact, or Monte-Carlo estimate methods are used to test the significance of the McNemar, marginal homogeneity, sign, and Wilcoxon tests.
- SPSS interface: These methods are available in SPSS, which can be accessed by selecting “analysis,” then “nonparametric,” and then the two related samples. Note that exact tests are available on “add on” module of SPSS.
McNemar test: This test is also called the symmetry McNemar chi-square test. The McNemar test variable can be nominal or ordinal. McNemar chi-square test tests whether or not the cell count above the diagonal differs from the count below the diagonal.
The McNemar test uses the chi-square distribution based on this formula:
Chi-square = (|b – c| – 1)2)/(b + c)
Degrees-of-freedom = (rows – 1) (columns – 1) = 1
McNemar test in SPSS is available in the analysis menu, by selecting “nonparametric,” and then clicking on the option called “two related samples.” Select “McNemar” and then select the row and column variable.
Interpreting McNemar test chi-square: If the computed value of McNemar test is less than the critical value using the degree of freedom, then it will not be significant. If the value shown by the SPSS output is less than the desired significant level, then the difference between the two dependent samples will be significant, otherwise, it will be non- significant.
The marginal homogeneity test: The marginal homogeneity test is much like the McNemar test, but in the marginal homogeneity test, the variables are dichotomous and in more than two categories. The marginal homogeneity test is interpreted like the McNemar test: if the p value is less than the desired significant value, then the two dependent sample means will be different, and if the p value is greater than the desired level, then the mean of the two dependent samples will be the same.
The sign test: The sign test is also a non-parametric test that is used when dependent samples are ordered in nature. The sign test converts the variable case into plus or minus signs and tests whether or not the plus sign differs from the minus sign. Based on the result, we can conclude if the two dependent sample means are the same or different. The sign test is considered a weaker test, because it tests the pair value below or above the median and it does not measure the pair difference. The sign test is available in SPSS: click “menu,” select “analysis,” then click on “nonparametric,” and choose “two related sample” and “sign test.”
Interpreting the sign test: If the p value of the sign test is less than the desired value, then the two dependent sample means will be different. If the p value of the sign test is more than the desired significant level, then the two sample means will be considered the same.
The Wilcoxon signed-rank test: This test is also called the Wilcoxon matched-pairs test when the dependent sample is interval or continuous in nature. The Wilcoxon signed-rank test is a more powerful test than the sign test, because it gives more information than the sign test. In SPSS, Wilcoxon signed-rank test is available by doing the following steps: in the menu click “analysis,” then select “nonparametric,” and choose “two related samples” and “Wilcoxon.” The SPSS output will also give information about the mean rank and the sum of ranks for the positive and negative sign. If the probability value is more than the desired value, than the Wilcoxon signed-rank test value will not be significant. If that is the case, we can say that the median rank of the two dependent samples is the same.
Assumptions:
- Data distribution: McNemar, Marginal Homogeneity, Sign, and Wilcoxon Tests are non parametric tests, so we do not assume that the data is normally distributed.
- Two sample: Data should be from two samples. The population may differ for the two samples.
- Dependent sample: Dependent samples should be a paired sample or before after sample.
- Adequate sample size: For the McNemar test, the number of the case should be equal to the a-d diagonal. If we are using the binomial test, then it should be equal to a+d diagonal.
Related Pages:
