The research methods section should reiterate the research questions and hypotheses, present the research design, discuss the participants, the instruments to be used, the procedure, the data analysis plan, and the sample size justification.
Research Questions and Null Hypotheses
In the research questions and null hypotheses section, the research questions should be restated in statistical language. For example, “Is there a difference in GPA by gender?” is a t-test type of question, whereas “Is there a relationship between GPA and income level?” is a correlation type of question. The important thing to remember is to use the language that foreshadows the data analysis plan. The null hypotheses are just the research questions stated in the null; for example, “There is no difference in GPA by gender,” or “There is no relationship between GPA and income level.”
The research design has several possibilities. First, you must decide if you are doing quantitative, qualitative, or mixed methods research. In a quantitative study, you are assessing participants’ responses on a measure. For example, participants can endorse their level of agreement on some scale. A qualitative design is a typically a semi-structured interview which gets transcribed, and the themes among the participants are derived. A mixed methods project is a mixture of both a quantitative and qualitative study.
In research methods, the participants are typically a sample of the population you want to study. You are probably not going to study all school children, but you may sample from the population of social children. You should probably speak about the characteristics of the population in your study (Are you sampling all males? teachers with under five years of experience?).
The instruments section is a critical research methods section. The instruments section should include the name of the instruments, the scales or subscales, how the scales are computed, and the reliability and validity of the scales. The instruments section should have references to the researchers who created the instruments.
The procedure section of the method is simply how you are going to administer the instruments that you just described to the participants you are going to select. You should walk the reader through the procedure in detail so that they can replicate your steps and your study.
Data Analysis Plan
The data analysis plan is just that — how you are going to analyze the data when you get the data from your participants. It includes the statistical tests you are going to use, the statistical assumptions of these tests, and the justification for the statistical tests.
Sample Size Justification/Power Analysis
Sample size justification (or power analysis) is selecting how many participants you need to have in your study. The sample size is based on several criteria: the power you select (which is typically .80), the alpha level selected (which is typically .05), and the effect size (typically, a large or medium effect size is selected). Importantly, once these criteria are selected, the sample size is going to be based on the type of statistic: an ANOVA is going to have a different sample size calculation than a multiple regression.
Research Methods Resources
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues for field settings. Boston, MA: Houghton-Mifflin. View
Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods approaches (2nd ed.). Thousand Oaks, CA: Sage Publications. View
Eamon, D. B. (1980). LABSIM: A data-driven simulation program for instruction in research design and statistics. Behavior Research Methods & Instrumentation, 12(2), 160-164.
Leedy, P., & Ormrod, J. E. (2004). Practical research: Planning and design (8th ed.). New York: Prentice Hall. View
Misangyi, V. F., LePine, J. A., Algina, J., & Goeddeke, F., Jr. (2006). The adequacy of repeated-measures regression for multilevel research: Comparisons with repeated-measures ANOVA, multivariate repeated-measures ANOVA, and multilevel modeling across various multilevel research designs. Organizational Research Methods, 9(1), 5-28.
Olejnik, S., & Algina, J. (2003). Generalized eta and omega squared statistics: Measures of effect size for some common research designs. Psychological Methods, 8(4), 434-447.
Pedhazur, E. J., & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Lawrence Erlbaum Associates. View
Proctor, R. W. (2005). Methodology is more than research design and technology. Behavior Research Methods, 37(2), 197-201.
Salazar, L. F., Crosby, R. A., & DiClemente, R. J. (2006). Choosing a research design. In R. A. Crosby, R. J. DiClemente, & L. F. Salazar (Eds.), Research methods in health promotion (pp. 2-23). San Francisco: Jossey-Bass.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton-Mifflin. View
Stone-Romero, E. F. (2002). The relative validity and usefulness of various empirical research designs. In S. G. Rogelberg (Ed.), Handbook of research methods in industrial and organizational psychology (pp. 77-98). Malden, MA: Blackwell Publishing.
Stone-Romero, E. F., & Rosopa, P. J. (2008). The relative validity of inferences about mediation as a function of research design characteristics. Organizational Research Methods, 11(2), 326-352.