Posted December 6, 2012

Its Christmas time again, and a good time to plan getting your dissertation completed, and that pesky scientific merit review, IRB application, concept paper, and dissertation proposal meeting. In the spirit of completing these tasks, there is a consistent problem that I find: graduate students struggle with the sample size determination. Part of the difficulty occurs because there are two types of sample sizes: Type I based on the population, and Type II based on the statistical power needed for a particular statistical test.

Type I. Let’s take the population sample size first. Dissertations rarely, if ever, use this type of sample size! Taking the sample size based on the population is to be able to generalize to that population. Well, if your studying 5^{th} grade teachers, there is no way (in terms of time, money, energy, stratification) you’re going to get a large enough sample to generalize to that population. If we had to generalize to populations for our dissertations, there would only be masters-level students out there—we’d just never finish. Now if you’re a business and really need to generalize to the population, feel free to call us and in 5 minutes we can help you determine this type of sample size.

Type II. This type of sample size justification IS used in dissertations, and is often referred to as a power analysis. It’s called a power analysis (usually .80 for most studies) because dissertation committees and universities want you to have an 80% chance of finding differences if in fact differences exist. This type of sample size is based on a number of factors, including, importantly, the statistical technique (the type of test, ANOVA, regression, correlation, etc) and effect size. The effect size is typically small, medium, or large.

Effect size. The second major roadblock to this sample size justification is the effect size (the first roadblock is making sure that the correct analysis is selected). There are theoretical and practical considerations to selecting this effect size. Theoretically, you need to look at other study’s that looked at the same phenomena or used the same measures and understand what effect size was used. Often times they will either present the effect size or it can be calculated from the statistics that are provided in the study. The other effect size consideration is practical. For a correlation with a small effect size you need 783 participants. If this is the case, we again might as well be happy with our master’s degree—it’s just really tough to get that many participants in some reasonable time. So then we’re in the large or medium effect size range, where for a correlation would be 28 (for a large effect) to 85 for a medium effect) participants. To determine sample size, you can mess with G-power or for $49 and the type of analysis in hand, can get a write-up of the sample size justification.

Wishing you all a wonderful Christmas and Hanukkah season. If you are conducting a study with sugar plums, you can decide what type of sample size is needed.

Best,

James Lani, Ph.D.