The basis of a multiple regression is to assess the relationship between one dependent variable and a set of independent or predictor variables. As the number of predictor variables increases, the amount of time to investigate the different equations also increases. Therefore, certain approaches are helpful in selecting and testing predictors to increase the efficiency of analysis.
The standard method of entry is simultaneous (a.k.a. the enter method); all variables are entered into the equation at the same time. This is an appropriate analysis when dealing with a small set of predictors and when the researcher does not know which variables will create the best prediction equation. Each predictor is assessed as though it were entered after all the other independent variables were entered, and assessed by what it offers to the prediction of the dependent variable that is different from the predictions offered by the other variables entered into the model.
Selection, on the other hand, allows for the construction of an optimal regression equation along with investigation into specific predictor variables. The aim of selection is to reduce the set of predictor variables to the number necessary that accounts for nearly as much of the variance as is accounted for by the total set. In essence, selection helps to determine the level of importance of each predictor variable. It also assists in assessing the effects once the other predictor variables are statistically eliminated. The circumstances of the study, along with the nature of the research questions guide the selection of predictor variables.
Four selection procedures are used to yield the most appropriate regression equation: forward selection, backward elimination, stepwise selection, and blockwise selection. The first three of these four procedures are considered statistical regression methods. Many times researchers use sequential regression (hierarchical or blockwise) entry methods that do not rely upon statistical results for selecting predictors. Sequential entry allows the researcher greater control of the regression process. Items are entered in a given order based on theory, logic or practicality and appropriate when the researcher has an idea as to how predictors may impact the dependent variable and how predictors are correlated with one another.
Statistical Regression Methods of Entry:
- Forward selection begins with an empty equation. Predictors are added one at a time beginning with the predictor with the highest correlation with the dependent variable. Variables of greater theoretical importance are entered first. Once in the equation, the variable remains there.
- Backward elimination (or backward deletion) is the reverse process. All the variables are entered into the equation first and each one is deleted one at a time if they do not contribute to the regression equation.
- Stepwise selection is considered a variation of the previous two methods. Stepwise selection involves analysis at each step to determine the contribution of the predictor variable entered previously in the equation. In this way it is possible to understand the contribution of the previous variables now that another variable has been added. Variables can be retained or deleted based on their statistical importance.
Sequential Regression Method of Entry:
- Blockwise selection is a version of forward selection that is achieved in blocks or sets. The predictors are grouped into blocks based on psychometric consideration or theoretical reasons and a stepwise selection is applied. Each block is applied separately while the other predictor variables are ignored. Variables can be removed when they do not contribute to the prediction. In general, the predictors included in the blocks will be intercorrelated. Also, the order of entry has an impact on which variables will be selected; those that are entered in the earlier stages have a better chance of being retained than those entered at later stages.
Essentially, the multiple regression selection process enables the researcher to obtain a reduced set of variables from a larger set of predictors, eliminating unnecessary predictors, simplifying data, and enhancing predictive accuracy. Two criterions are used to achieve the best set of predictors; these include meaningfulness to the situation and statistical significance. By entering variables into the equation in a given order, confounding variables can be investigated and variables that are highly correlated can be combined into blocks.
References:
Cohen, J. & Cohen, P. (1983). Applied multiple regression/correlation analysis for the
behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
Cramer, D. (1998). Fundamental statistics for social research: Step by step calculations and
computer techniques using SPSS for Window. New York, NY: Routledge.
Halinski, R. S. & Feldt, L. S. (1970). The selection of variables in multiple regression analysis.
Journal of Educational Measurement, 7 (3). 151-157.
Leech, N. L., Barrett, K. C., & Morgan, G.A. (2008). SPSS for intermediate statistics: Use and
interpretation (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.
Pedhazur, E. (1997). Multiple regression in behavioral research: Explanation and prediction (3rd
ed.). Orlando, FL: Holt, Rinehart & Winston, Inc.
Stevens, J. P. (2002). Applied multivariate statistics for the social sciences (4th ed.). Mahwah,
NJ: Lawrence Erlbaum Associates.
Tabachnick, B. G. & Fidell, L. S. (2001). Using multivariate statistics (4th ed.). Boston, MA:
Allyn and Bacon.
