Comprehensive Guide to Factor Analysis

Introduction to Factor Analysis

Factor analysis is a sophisticated statistical method aimed at reducing a large number of variables into a smaller set of factors. This technique is valuable for extracting the maximum common variance from all variables, transforming them into a single score for further analysis. As a part of the general linear model (GLM), factor analysis is predicated on certain key assumptions such as linearity, absence of multicollinearity, inclusion of relevant variables, and a true correlation between variables and factors.

Principal Methods of Factor Extraction

Principal Component Analysis (PCA):

PCA is the most widely used technique. It begins by extracting the maximum variance, assigning it to the first factor. Subsequent factors are determined by removing variance accounted for by earlier factors and extracting the maximum variance from what remains. This sequential process continues until all factors are identified.

Common Factor Analysis:

Preferred for structural equation modeling (SEM), this method focuses on extracting common variance among variables, excluding unique variances. It’s particularly useful for understanding underlying relationships that may not be immediately apparent from the observed variables.

Image Factoring:

Based on a correlation matrix, image factoring uses ordinary least squares regression to predict factors, making it distinct in its approach to factor extraction.

Maximum Likelihood Method:

This technique utilizes the maximum likelihood estimation approach to factor analysis, working from the correlation matrix to derive factors.

Other Methods:

Including Alpha factoring and weighted least squares, these methods provide alternatives that may be suitable depending on the specific characteristics of the data set.

Factor Loadings and Their Interpretation

Factor loadings play a crucial role in factor analysis, representing the correlation between the variable and the factor. A factor loading of 0.7 or higher typically indicates that the factor sufficiently captures the variance of that variable. These loadings help in determining the importance and contribution of each variable to a factor.

Looking for assistance with your research?

Schedule a time to speak with an expert using the calendar below.

User-friendly Software

Transform raw data to written interpreted APA formatted results in seconds.

Watch and Get Access

Eigenvalues and Factor Scores

  • Eigenvalues: Also known as characteristic roots, eigenvalues represent the variance explained by a factor out of the total variance. They are critical for understanding the contribution of each factor to explaining the pattern in the data.
  • Factor Scores: These scores, which can be standardized, represent the estimated scores of each observation for the factors and are used for further analysis. They essentially provide a way to reduce the dimensionality of the data set while retaining as much information as possible.

Determining the Number of Factors

The number of factors to retain can be determined by several criteria:

  • Kaiser Criterion: An eigenvalue greater than one suggests that the factor should be retained.
  • Variance Extraction Rule: Factors should explain a significant portion of the variance, typically set at a threshold of 0.7 or higher.

Rotation Techniques to Enhance Interpretability

Rotations in factor analysis, whether orthogonal like Varimax and Quartimax or oblique like Direct Oblimin and Promax, help in achieving a simpler, more interpretable factor structure. These methods adjust the axes on which factors are plotted to maximize the distinction between factors and improve the clarity of the results.

Assumptions and Data Requirements

  • Data Characteristics: Factor analysis assumes no outliers, a sufficient sample size (cases should exceed the number of factors), and interval-level data measurement.
  • Statistical Assumptions: There should be no perfect multicollinearity among variables, and while the model assumes linearity, nonlinear variables can be transformed to meet this requirement.

Conclusion

Factor analysis is a powerful tool for data reduction and interpretation, enabling researchers to uncover underlying dimensions or factors that explain patterns in complex data sets. By adhering to its assumptions and appropriately choosing factor extraction and rotation methods, researchers can effectively use factor analysis to simplify data, construct scales, and enhance the validity of their studies.

Resources

Bryant, F. B., & Yarnold, P. R. (1995). Principal components analysis and exploratory and confirmatory factor analysis. In L. G. Grimm & P. R. Yarnold (Eds.), Reading and understanding multivariate analysis. Washington, DC: American Psychological Association.

Dunteman, G. H. (1989). Principal components analysis. Newbury Park, CA: Sage Publications.

Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272-299.

Gorsuch, R. L. (1983). Factor Analysis. Hillsdale, NJ: Lawrence Erlbaum Associates.

Hair, J. F., Jr., Anderson, R. E., Tatham, R. L., & Black, W. C. (1995). Multivariate data analysis with readings (4th ed.). Upper Saddle River, NJ: Prentice-Hall.

Hatcher, L. (1994). A step-by-step approach to using the SAS system for factor analysis and structural equation modeling. Cary, NC: SAS Institute.

Hutcheson, G., & Sofroniou, N. (1999). The multivariate social scientist: Introductory statistics using generalized linear models. Thousand Oaks, CA: Sage Publications.

Kim, J. -O., & Mueller, C. W. (1978a). Introduction to factor analysis: What it is and how to do it. Newbury Park, CA: Sage Publications.

Kim, J. -O., & Mueller, C. W. (1978b). Factor Analysis: Statistical methods and practical issues. Newbury Park, CA: Sage Publications.

Lawley, D. N., & Maxwell, A. E. (1962). Factor analysis as a statistical method. The Statistician, 12(3), 209-229.

Levine, M. S. (1977). Canonical analysis and factor comparison. Newbury Park, CA: Sage Publications.

Pett, M. A., Lackey, N. R., & Sullivan, J. J. (2003). Making sense of factor analysis: The use of factor analysis for instrument development in health care research. Thousand Oaks, CA: Sage Publications.

Shapiro, S. E., Lasarev, M. R., & McCauley, L. (2002). Factor analysis of Gulf War illness: What does it add to our understanding of possible health effects of deployment, American Journal of Epidemiology, 156, 578-585.

Velicer, W. F., Eaton, C. A., & Fava, J. L. (2000). Construct explication through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factors or components. In R. D. Goffin & E. Helmes (Eds.), Problems and solutions in human assessment: Honoring Douglas Jackson at seventy. Boston, MA: Kluwer.

Widaman, K. F. (1993). Common factor analysis versus principal component analysis: Differential bias in representing model parameters, Multivariate Behavioral Research, 28, 263-311.

Related Pages:

Statistics Solutions can assist with your quantitative analysis by assisting you to develop your methodology and results chapters. The services that we offer include:

Data Analysis Plan

  • Edit your research questions and null/alternative hypotheses
  • Write your data analysis plan; specify specific statistics to address the research questions, the assumptions of the statistics, and justify why they are the appropriate statistics; provide references
  • Justify your sample size/power analysis, provide references
  • Explain your data analysis plan to you so you are comfortable and confident
  • Two hours of additional support with your statistician

Quantitative Results Section (Descriptive Statistics, Bivariate and Multivariate Analyses, Structural Equation Modeling, Path analysis, HLM, Cluster Analysis)

Clean and code dataset

  • Conduct descriptive statistics (i.e., mean, standard deviation, frequency and percent, as appropriate)
  • Conduct analyses to examine each of your research questions
  • Write-up results
  • Provide APA 7th edition tables and figures
  • Explain Chapter 4 findings
  • Ongoing support for entire results chapter statistics

Please call 727-442-4290 to request a quote based on the specifics of your research, schedule using the calendar on this page, or email [email protected]