Core Concepts & Assumptions of Multiple Linear Regression: The Foundation of Your Analysis

Detailed Breakdown of MLR Assumptions

Adherence to the assumptions of MLR ensures that the ordinary least squares (OLS) estimation method yields the best linear unbiased estimators (BLUE). Violations can lead to misleading or incorrect conclusions.

1. Linearity: The relationship between each independent variable and the dependent variable must be linear.
   1. How to check: This is typically assessed by visually examining scatterplots of each independent variable against the dependent variable, or by examining a scatterplot of residuals versus predicted values. A random scatter of residuals around zero supports linearity.
   1. Consequences of violation: If the true relationship is non-linear and a linear model is fitted, the regression coefficients may be biased, and the model will not accurately represent the data.
   1. Intellectus Advantage: Software like Intellectus Statistics can facilitate the visualization of these relationships through automated plot generation, making the assumption checking process more straightforward and less prone to oversight.
2. Independence of Observations/Errors: The observations (and, more critically, their errors or residuals) must be independent of each other. This means that the error term for one observation should not be correlated with the error term of another.
   1. How to check: The Durbin-Watson statistic is commonly used to detect autocorrelation (a common violation in time-series data). For other types of dependencies, such as data clustered within groups (e.g., students within classrooms), understanding the study design is key. Such data may require multilevel modeling rather than standard MLR.
   1. Consequences of violation: While regression coefficients might remain unbiased, their standard errors are likely to be underestimated, leading to inflated t-statistics and an increased risk of Type I errors (false positives). The estimates will also be inefficient.
3. Homoscedasticity (Constant Variance of Errors): The variance of the residuals (errors) should be constant across all levels of the independent variables (or across all predicted values).
   1. How to check: This is assessed by visually examining a scatterplot of the standardized residuals against the standardized predicted values. The plot should show a random, horizontal band of points; a fanning or cone shape suggests heteroscedasticity.
   1. Consequences of violation: OLS estimates remain unbiased, but they are no longer the most efficient. Standard errors become biased, invalidating t-tests and F-tests.
4. Normality of Residuals: The residuals of the regression model should be approximately normally distributed. It is important to note that it is the errors that are assumed to be normally distributed, not necessarily the raw variables themselves.
   1. How to check: This can be examined using histograms, P-P plots (Probability-Probability plots), or Q-Q plots (Quantile-Quantile plots) of the residuals, or statistical tests like the Shapiro-Wilk or Kolmogorov-Smirnov test (though visual inspection is often preferred, especially with larger samples).
   1. Consequences of violation: For small sample sizes, non-normality of residuals can affect the validity of p-values and confidence intervals for the coefficients. MLR is fairly robust to violations of this assumption with larger sample sizes due to the Central Limit Theorem.
5. No Perfect Multicollinearity (and low problematic multicollinearity): Independent variables should not be perfectly correlated with each other. While perfect multicollinearity is rare, high multicollinearity (where independent variables are very strongly correlated) is a more common and problematic issue.
   1. How to check: Examine the correlation matrix of predictors for high correlations (e.g., > 0.8 or 0.9). More formally, check Tolerance values (values close to 0, e.g., < 0.10, indicate a problem) and Variance Inflation Factor (VIF) values (Tolerance = 1/VIF). VIF values greater than 5 are often considered indicative of moderate multicollinearity, while VIFs greater than 10 suggest serious multicollinearity that needs addressing.
   1. Consequences of violation: Multicollinearity inflates the standard errors of the regression coefficients, making them unstable and difficult to interpret. It becomes challenging to assess the individual contribution of each correlated predictor to the model, and coefficients might even have unexpected signs or magnitudes. The overall predictive power of the model (R-squared) may remain high, but the individual predictor effects are unreliable.
6. No Significant Outliers or Highly Influential Points: Outliers are observations with extreme values on the dependent or independent variables, while influential points are those that disproportionately affect the regression model’s parameters.
   1. How to check: Outliers can be detected using casewise diagnostics, standardized residuals (e.g., values > |3.29|), or studentized deleted residuals. Influential points can be identified using measures like Cook’s Distance or Leverage values.
   1. Consequences of violation: Outliers and influential points can severely distort the estimated regression coefficients and lead to a model that does not accurately reflect the underlying relationships in the majority of the data.

Dissertation committees expect a thorough check of these assumptions. Providing evidence that assumptions have been met, or that violations have been appropriately addressed, lends credibility and rigor to the statistical analysis chapter.