# Regression

A regression assesses whether predictor variables account for variability in a dependent variable.  This page will describe regression analysis examples, regression assumptions, the evaluation of the R-square (coefficient of determination), the F-test, the interpretation of the beta coefficient(s), and the regression equation.

Do age and gender impact attitudes on gun regulation?

How do the five facets of mindfulness influence intent to purchase?

Example

There are numerous applications of regression.  For example, a school has two types of reading programs (tradition program and a novel program), and would like to examine if program type influences (predicts) test scores.  Assumptions

First, this analysis is very sensitive to outliers.   The researcher should first standardize the scores to see if a value is +/- 3.29 standard deviations from the other scores and consider deleting that score.

Second, the main assumptions of regression are linearity and constant variance.   To assess assumptions the researcher should plot the standardized residuals verses the predicted values.  If the plot shows random scatter (homoscedasticity), the assumptions are met.  However, if there is a curvilinear shape (e.g., U-shape) then linearity is not met, or if the scatter has a cone shape, then constant variance of the regression analysis is not met.

F-test

When the regression is conducted, a F-value and significance level of that F-value will be in the output.  If the F-value is statistically significant (typically p < .05), this signifies that the model (the predictors) did a good job of predicting the outcome variable and that there is a significant relationship between the set of predictors and the dependent variable

Evaluation of the R-Square

When the regression is conducted, an R2 (coefficient of determination) is presented. This value is the multivariate equivalent of the bivariate correlation coeffieicent.  The R2 really answers the question, “of all of the reasons why the outcome variable can vary, what percent of those reasons can be accounted for by the predictor(s) variables.”

The regression output will also present an adjusted R2 value.  The interpretation of this value is that if the researcher used this model on a new data set, this would be the amount of variability accounted for in the new data set.  Sample size differences between data sets would reason to interpret the adjusted R2 value.

Beta Coefficients

After the evaluation of the F-value and R2, it is important to evaluate the regression beta coefficients: unstandardized and standardized.  The beta coefficients can be negative or positive, and have a t-value and significance of that t-value associated with it.  Think of the regression beta coefficient as the slope of a line: the t-value and significance assesses the extent to which the magnitude of the slope is significantly different from the line laying on the X-axis.  If the beta coefficient is not statistically significant (i.e., the t-value is not significant), no statistical significance can be interpreted from thatpredictor.  If the beta coefficient is sufficient, examine the sign of the beta. If the regression beta coefficient is positive, the interpretation is that for every 1-unit increase in the predictor variable, the dependent variable will increase by the unstandardized beta coefficient value.  For example, if the beta coefficient is .80 and statistically significant, then for each unit increase in the predictor variable, the outcome variable will increase by .80 units.

Equation

Once the beta coefficient is determined, then a regression equation can be written.  Using the example and beta coefficient above, the equation can be written as follows:

Y=.80X + c, where Y is the outcome variable, X is the predictor variable, .80 is the beta coefficient, and c is a constant.