Yesterday, we went over the basics of sample size calculation. Now, let’s move on to individual tests. Since sample size calculation depends on the test, it’s crucial to determine the minimum sample size based on the type of statistical analysis being conducted. Today, I will cover sample size calculation for independent samples t-tests.
Power Analysis for Independent Samples t-tests
For this discussion, I will refer to Jacob Cohen’s short journal article titled Quantitative Methods in Psychology: A Power Primer. This insightful journal article not only identifies effect sizes for common statistical tests but also provides the necessary sample sizes to achieve significance for those tests. Specifically, it considers different levels of significance (0.01, 0.05, and 0.10), various effect sizes (small, medium, and large), and a power of 0.80. Furthermore, it serves as a useful resource for understanding how these factors interrelate in the context of statistical analysis.
Key Factors for Sample Size Calculation in an Independent Samples t-Test
Yesterday we covered the facets of sample size calculation and actually used the example of an independent samples t-test. We need a couple things to calculate the sample size necessary for our t-test:
· Level of significance: The probability of committing a Type I error, or falsely rejecting the null hypothesis.Usually this is 0.05, making the probability of committing a Type I error 5%.We decide this.
· Estimated effect size: For simplicity purposes, we are going to think of this as the difference between the two groups. This is an estimate and will be determined by our software for the samples we are using. We have no control over this.
· Power: We also consider the probability of committing a Type II error, which is the chance of falsely accepting the null hypothesis. Typically, this probability is set at 0.80, meaning the likelihood of committing a Type II error is 20%, or four times greater than the probability of committing a Type I error. Furthermore, this value is something we determine as well.
Schedule a time to speak with expert using the link below
A priori Sample Size for Independent Samples t-tests
Of all the sample size calculations, this is probably the easiest. To assist with this, Page 157 of Quantitative Methods in Psychology: A Power Primer tabulates effect sizes for common statistical tests. Specifically, the first entry is the t-test for the difference between two independent means, also known as the independent samples t-test. It tells us that a small effect size is 0.20, a medium effect size is 0.50, and a large effect size is 0.80.
Ideally, the effect size should come from empirical research. However, for this calculation, we’ll use a 0.05 level of significance, estimate a medium effect size of 0.50, and aim for a power of 0.80. Based on these criteria, Page 158 shows that we need 64 participants in each group, totaling 128 participants.