Validity
Validity implies precise and exact results acquired from the data collected. In technical terms, a measure can lead to a proper and correct conclusion and result from a sample that can be taken as a valid conclusion about the population.
Validity has four major types. They are as follows:
1. Internal validity: Internal validity is when the relationship between variables is causal. Internal validity refers to the relationship between dependent and independent variables. Internal validity is associated with the design of the experiment and is only relevant in studies that try to establish a causal relationship. Internal validity, for example, can be used for the random assignment of treatments.
2. External validity: When there is a causal relationship between the cause and effect that can be transferred to people, treatments, variables, and different measurement variables which differ from the other, this is called external validity.
3. Statistical conclusion validity: Statistical conclusion validity is the conclusion reached or inference drawn about the extent of the relationship between the two variables. For instance, it can be found when we aim at finding the strength of relationship between any two variables that have been under observation and analysis. If we do reach the correct conclusion, then it is said to be statistical conclusion validity. There are two types of statistical conclusion validity. They are as follows:
a. Type one error: Type one error is when we conclude that there is a relationship between two variables and we accept a hypothesis when in reality, there is no relationship between the two variables. This is in fact very dangerous.
b. Type two errors: If we reject a hypothesis that is true it is called type two error.
In statistical conclusion validity, the method of power analysis is used to detect the relationship. Several problems crop up while making statistical conclusion validity. For instance, if a small sample size is used, then there is the possibility that the result will not be correct. To avoid this, the sample size should be of considerable size. Statistical validity is also threatened by the violation of statistical assumptions. The results may not be accurate, however, if values in analysis are biased and the wrong statistical test is approved.
4. Construct validity: When construct is used for predicating the relationship for the dependent variable, it implies construct validity. For instance, in structural equation modeling, when we draw the construct, then we presume that the factor loading for the construct is greater than .7. To draw construct validity, Cronbach’s alpha is used. For exploratory purposes .60 is accepted, for confirmatory purposes .70 is accepted, and .80 is considered good. If the construct satisfies the above presumption and expectation, then the construct would be helpful in predicting the relationship for dependent variables. Convergent/divergent validation and factor analysis is also used to test construct validity.
Relationship between reliability and validity: There is no way that a test that is unreliable is valid. Again, any test that is valid must be reliable. By this statement we are able to derive that validity plays a significant role in analysis as it ensures the conclusion of accurate results. Reliability is equally important, but it is not a sufficient condition for validity.
Overall validity threats:
1. Insufficient data collected to make valid conclusion
2. Measurement done with too few measurement variables
3. Too much variation in data or outlier in data
4. Wrong selection of samples
5. Inaccurate measurement method taken for analysis
Validity Resources
Bagozzi, R. P., Yi, Y., & Phillips, L. W. (1991). Assessing construct validity in organizational research. Administrative Science Quarterly, 36(3), 421-458.
Brinkman, W. -P., Haakma, R., & Bouwhuis, D. G. (2009). The theoretical foundation and validity of a component-based usability questionnaire. Behaviour & Information Technology, 28(2), 121-137.
Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Thousand Oaks, CA: Sage Publications.
Cronbach, L. J. (1971). Test validation. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 443-507). Washington, DC: American Council on Education.
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39-50.
Guilford, J. P. (1946). New standards for test evaluation. Educational and Psychological Measurement, 6(5), 427-439.
Krause, M. S. (1972). The implications of convergent and discriminant validity data for instrument validation. Psychometrika, 37(2), 179-186.
Lieberman, D. Z. (2008). Evaluation of the stability and validity of participant samples recruited over the internet. CyberPsychology & Behavior, 11(6), 743-746.
Lozano, L. M., Carcía-Cueto, E., & Muñoz, J. (2008). Effect of the number of response categories on the reliability and validity of rating scales. Methodology, 4(2), 73-79.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Education measurement (3rd ed., pp. 13-103). Washington, DC: American Council on Education.
Moret, M., Reuzel, R., van der Wilt, G. J., & Grin, J. (2007). Validity and reliability of qualitative data analysis: Interobserver agreement in reconstructing interpretative frames. Field Methods, 19(1), 24-39.
Rosenbaum, P. R. (1989). Criterion-related construct validity. Psychometrika, 54(4), 625-659.
Shepard, L. A. (1993). Evaluating test validity. Review of Research in Education, 19, 405-450.
