Sample Size Calculation: A Few Considerations

An important aspect of writing your methodology is determining an appropriate sample size. While it is a good idea to review past studies to see what previous researchers have estimated and achieved in terms of sample size, you should base your sample size estimations on your unique study parameters and what is feasible in terms of your population and procedures. While sample size calculations can be done by hand, I recommend using a specifically-designed calculator for ease of use and reduced user error. A good comprehensive software to use is G*Power, developed by Faul, Buchner, Erdfelder, and Lang (click here to check it out).

There are a few major elements that go into most sample size (or power) calculations: the type of analysis you plan to conduct, the estimated effect size, the alpha level (i.e., your p-value or significance level), your desired power level, and a few other parameters specific to the type of analysis (e.g., number of groups in an ANOVA or number of predictors in a regression). The type of analysis you plan to conduct should come from your research questions and the type of data you plan to collect. There are commonly accepted values for your alpha and power levels, which come from balancing Type I and II errors (click here for more on Type I and II errors). Commonly accepted alpha levels are .05 and the more stringent .01, which represent a 5% and 1% chance of committing a Type I error, respectively. A commonly acceptable power level in social science research is .80.

Choosing what effect size to input is a bit trickier. To be the most accurate, you will want to see what effect size similar studies have achieved. But, what if there are no studies similar to yours, or they did not report their effect sizes? In that case, you will want to decide what the minimum effect size is that you would want to “detect.” For example, if you are running a study with medical implications, where even a small effect would have clinical significance, calculate your sample based on a small effect. G*Power makes it easy to determine the value of the appropriate effect size, as it provides numbers for what would be considered a small, medium, or large effect for several common analyses. After that, all you need to do is to input the parameters specific to your analysis (such as number of predictor variables) and calculate your sample size.

Keep in mind that this calculation just gives you the ideal sample size based on the parameters of your analysis. This does not mean that you are guaranteed to find significant results with the calculated sample size. For example, if you calculated a sample size of 60 participants using .05 alpha, .80 power, and a medium effect size, this does not mean that recruiting 60 participants guarantees that you will find a significant result. If fact, assuming your estimate of the effect size was perfectly accurate, you would still have about a 20% chance to not detect that effect.

If you get an estimated sample size that is unreasonable for your specific population or procedures, there are several things you can do, including simplifying your analysis, estimating a larger effect size, or accepting reduced power as a study limitation. Because more complex statistical analyses often require more participants, if possible, try to simplify your analysis. For example, if you are conducting a multiple regression, you can reduce the number of predictors. This will lower your sample size requirement. Another alternative is to search the literature to see if you can find justification for using a larger effect size in your calculation. Larger effect sizes are easier to detect and thus require fewer participants. However, there is no guarantee that you will find the same size effect (or any effect at all) in your own sample.

If you cannot reduce your model, cannot find previous literature that supports a larger effect size, and you know you are unlikely to reach your sample size estimation because of a rare population or limited resources, you might have to accept reduced statistical power as a limitation. Now, a drastically small sample size is not recommended for conducting most inferential statistics, but you may be able to propose your study as exploratory or descriptive research and recommend that future research look into increased recruitment measures. Even if you are not able to reach it, calculating your sample size will put you in a more informed and prepared position to interpret and understand your results.