Discriminant analysis is a statistical technique that is used to classify the dependent variable between two or more categories. Discriminant analysis also has a regression technique, which is used for predicting the value of the dependent categorical variable.
In discriminant analysis, we predict the value of two categories. When the category of a dependent variable is more than two, it will simply be an extension of the simple discriminant analysis called the multiple discriminant analysis. Multiple discriminant analysis and MANOVA are considered to be similar because both tests share many similar assumptions and tests.
The F test (Wilks’ lambda) is used to test whether or not the discriminant model is significant as a whole. If the F test shows the overall significance of the model, then the individual variables are accessed to see which variable will move the significance from the group mean.
Discriminant analysis also has several assumptions, such as multiple linear regressions, linear relationships, homoscedastic relationships, untruncated interval data, etc. Logistic regression is the alternative technique and it is frequently used in place of discriminant analysis when data does not meet the assumptions.
Key Terms and Concepts:
Discriminating variables: Discriminating variables are independent variables that are used to predict the dependent variable. These variables are also called the predictors.
The criterion variable: Dependent variables are also called the criterion variables.
Discriminant function: The Linear combination of the discriminating (independent) variable is called the discriminant function. For example,
L = b1×1 + b2×2 + … + bnxn + c
where L= discriminant function, b1= discriminant coefficients, X= independents variables, and C = constants
Number of discriminant functions: For the two groups, there is one discriminant analysis function. For multivariate discriminant analysis there will be g-1 discriminant function.
The Eigenvalues: This is also called characteristic root, which tells us the variance explained by each discriminant function.
The discriminant score: By applying discriminant formulas, the value that comes is called the discriminant score. This discriminant score helps us to classify the group category.
Cutoff: This is the value which divides the group value into two parts. When the value of the discriminant score is at the negative side of the cutoff point, then the group will fall into a lower category, and when it is at the positive side, the group will be at a higher category.
Unstandardized discriminant coefficients: Unstandardized discriminant coefficients are simply like the regression beta, which is used to predict the discriminate score. Standardized discriminant coefficients are used to compare the relative importance of the independent variables.
Tests of significance:
Wilks’ lambda: The overall model significance of the discriminant function is tested by the Wilks’ lambda test. If the overall model is significant, then the F test is used to test whether or not the individual variable means differ from the group mean function.
Assumptions in Discriminant analysis:
1. Independence: Each case should be independent of each other. Correlated data cannot be used in discriminant analysis.
2. Adequate sample size: There must be at least two cases for each category of the dependent variable. However, it is recommended that there should be at least four or five times as many cases as independent variables.
3. Interval data: In discriminant analysis, there should be an interval data for independent variable.
4. Variance: No independents have a zero standard deviation in one or more of the groups formed by the dependent.
5. Random error: Error terms are assumed to be randomly distributed.
6. Homogeneity of variances: Variance with each group of independent variables should be equal.
7. Absence of perfect multicollinearity: There should be no perfect multicollinearity between the independent variables.
8. Assumes linearity: The discriminant functions should be linear and related to each other.
9. Normally distributed: The predictor variable should be normally distributed.
Discriminant Analysis Resources
Burr, T., & Doak, J. (2007). Distribution-free discriminant analysis. Intelligent Data Analysis, 11(6), 651-662.
Croux, C., & Joossens, K. (2005). Influence of observations on the misclassification probability in quadratic discriminant analysis. Journal of Multivariate Analysis, 96(2), 348-403.
Dunteman, G. H. (1984). Introduction to multivariate analysis. Thousand Oaks, CA: Sage Publications.
Finch, H., & Schneider, M. K. (2007). Classification accuracy of neural networks vs. discriminant analysis, logistic regression, and classification and regression trees: Three- and five-group cases. Methodology, 3(2), 47-57.
Huberty, C. J. (1994). Applied discriminant analysis. New York: John Wiley & Sons.
Huberty, C. J., & Hussein, M. H. (2003). Some problems in reporting use of discriminant analyses. Journal of Experimental Education, 71(2), 177-191.
Klecka, W. R. (1980). Discriminant analysis. Newbury Park, CA: Sage Publications.
Lachenbruch, P. A., & Goldstein, M. (1979). Discriminant Analysis. Biometrics, 35, 69-85.
Lee, S. -Y., Song, X. -Y., & Lu, B. (2007). Discriminant analysis using mixed continuous, dichotomous, and ordered categorical variables. Multivariate Behavioral Research, 42(4), 631-645.
McLachlan, G. J. (2004). Discriminant analysis and statistical pattern recognition. New York: John Wiley & Sons.
Morgan, G. A., Vaske, J. J., Gliner, J. A., & Harmon, R. J. (2003). Logistic regression and discriminant analysis: Use and interpretation. Journal of the American Academy of Child & Adolescent Psychiatry, 42(8), 994-997.
Press, S. J., & Wilson, S. (1978). Choosing between logistic regression and discriminant analysis. Journal of the American Statistical Association, 73(364), 699-705.
Shin, H. (2008). An extension of Fisher’s discriminant analysis for stochastic processes. Journal of Multivariate Analysis, 99(6), 1191-1216.
