Regression

A regression assesses whether predictor variables account for variability in a dependent variable.  This page will describe regression analysis example research questions, regression assumptions, the evaluation of the R-square (coefficient of determination), the F-test, the interpretation of the beta coefficient(s), and the regression equation.

Example questions answered by a regression analysis:

Do age and gender predict gun regulation attitudes?

Do the five facets of mindfulness influence peace of mind scores?

 

Assumptions:

First, regression analysis is sensitive to outliers.  Outliers can be identified by standardizing the scores and checking the standardized scores for absolute values higher than 3.29.  Such values may be considered outliers and may need to be removed from the data.

Second, the main assumptions of regression are normality, homoscedasticity, and absence of multicollinearity.  Normality can be assessed by examining a normal P-P plot.  If the data form a straight line along the diagonal, then normality can be assumed.  To assess homoscedasticity, the researcher can create a scatterplot of standardized residuals verses standardized predicted values.  If the plot shows random scatter, the assumption is met.  However, if the scatter has a cone shape, then the assumption is not met.  Multicollinearity can be assessed by calculated variance inflation factors (VIFs).  VIF values higher than 10 indicates that multicollinearity may be a problem.

F-test

When the regression is conducted, an F-value, and significance level of that F-value, is computed.  If the F-value is statistically significant (typically p < .05), the model explains a significant amount of variance in the outcome variable.

Evaluation of the R-Square

When the regression is conducted, an R2 statistic (coefficient of determination) is computed.  The R2 can be interpreted as the percent of variance in the outcome variable that is explained by the set of predictor variables.

Evaluation of the Adjusted R-Square

The adjusted R2 value is calculation of the R2 that is adjusted based on the number of predictors in the model.

Beta Coefficients

After the evaluation of the F-value and R2, it is important to evaluate the regression beta coefficients.  The beta coefficients can be negative or positive, and have a t-value and significance of the t-value associated with each.  The beta coefficient is the degree of change in the outcome variable for every 1-unit of change in the predictor variable.  The t-test assesses whether the beta coefficient is significantly different from zero.  If the beta coefficient is not statistically significant (i.e., the t-value is not significant), the variable does not significantly predict the outcome.  If the beta coefficient is significant, examine the sign of the beta.  If the beta coefficient is positive, the interpretation is that for every 1-unit increase in the predictor variable, the outcome variable will increase by the beta coefficient value.  If the beta coefficient is negative, the interpretation is that for every 1-unit increase in the predictor variable, the outcome variable will decrease by the beta coefficient value.  For example, if the beta coefficient is .80 and I statistically significant, then for each 1-unit increase in the predictor variable, the outcome variable will increase by .80 units.

Equation

Once the beta coefficient is determined, then a regression equation can be written.  Using the example and beta coefficient above, the equation can be written as follows:

y= 0.80x + c, where y is the outcome variable, x is the predictor variable, 0.80 is the beta coefficient, and c is a constant.

*For assistance with conducting regressions or other quantitative analyses click here.

Related Pages:

Linear Regression
Multiple Linear Regression
Logistic Regression
Ordinal Regression