바로가기메뉴

본문 바로가기 주메뉴 바로가기

Korean Journal of Psychology: General

A multi-group analysis using the categorical confirmatory factor analysis

Korean Journal of Psychology: General / Korean Journal of Psychology: General, (P)1229-067X; (E)2734-1127
2020, v.39 no.2, pp.175-204
https://doi.org/10.22257/kjp.2020.6.39.2.175


Abstract

Categorical confirmatory factor analysis (CFA) is a measurement model that incorporates the categorical nature of the Likert scale, which is frequently used in the social sciences, including but not limited to psychology. The categorical CFA has quite different and complex model specification and identification as compared to the typical continuous CFA. A multi-group extension of the categorical CFA needs even more restrictions to be properly specified and identified. Moreover, the steps in checking the measurement invariance with the categorical indicator variables differ from those with the case of continuous indicator variables. The present study aims to help researchers choose proper methods and procedures for using a multi-group categorical CFA by investigating and integrating unorganized existing literature. To achieve this goal, we focus on two aspects. Based on the limited information estimation, we introduce scaling methods for the two types of latent variables, latent response variables and factors, and also address the method for parameter restrictions for multi-group analysis. Next, focusing on threshold parameters of the categorical CFA, we provide possible procedures for checking the measurement invariance. We then illustrate an application of the whole procedures using real data, and finally discuss the results.

keywords
범주형 자료, 다집단 분석, 요인분석, 제한정보추정, 측정불변성, categorical data, multi-group analysis, factor analysis, limited information estimation, measurement invariance

Reference

1.

Asparouhov, T., & Muthén, B., (2006). Robust chi square difference testing with mean and variance adjusted test statistics. Matrix, 1(5), 1-6.

2.

Asparouhov, T., & Muthén, B. (2010). Simple second order chi-square correction. Unpublished manuscript.

3.

Asparouhov, T., & Muthén, B. (2014). Auxiliary variables in mixture modeling: Three-step approaches using M plus. Structural Equation Modeling, 21(3), 329-341.

4.

Baas, K. D., Cramer, A. O., Koeter, M. W., van de Lisdonk, E. H., van Weert, H. C., &Schene, A. H. (2011). Measurement invariance with respect to ethnicity of the Patient Health Questionnaire-9 (PHQ-9). Journal of Affective Disorders, 129(1-3), 229-235.

5.

Batinic, B., Wolff, H. G., & Haupt, C. M. (2008). Construction and Factorial Structure of a Short Version of the Trendsetting Questionnaire (TDS-K) A Cross-Validation Using Multigroup Confirmatory Factor Analyses. European Journal of Psychological Assessment, 24(2), 88-94.

6.

Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling, 13(2), 186-203.

7.

Bernstein, I. H., & Teng, G. (1989). Factoring items and factoring scales are different:Spurious evidence for multidimensionality due to item categorization. Psychological Bulletin, 105(3), 467-477.

8.

Bollen, K. A. (1989). A new incremental fit index for general structural equation models. Sociological Methods & Research, 17(3), 303-316.

9.

Byrne, B. M. (2012). Stuctural equation modeling with Mplus: Basic concepts, applications, and programming. New York, NY: Taylor & Francis Group.

10.

Byrne, B. M., Shavelson, R. J.. & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial meausrement invariance. Psychological Bulletin, 105(3), 456-466.

11.

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement Invariance. Structural Equation Modeling, 14(3), 464-504.

12.

Cheung, G. W., & Rensvold, R. B. (2000). Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling. Journal of Cross-Cultural Psychology, 31(2), 187-212.

13.

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233-255.

14.

Choi, S. & Cho, Y. (2013). Measurement invariance of self-esteem between male and female adolescents and method effects associated with negatively worded items. Korean Journal of Psychology: General, 32(3), 571-589.

15.

Christoph, G., Goldhammer, F., Zylka, J., &Hartig, J. (2015). Adolescents' computer performance: The role of self-concept and motivational aspects. Computers & Education, 81, 1-12.

16.

Cooke, D. J., Kosson, D. S., & Michie, C. (2001). Psychopathy and ethnicity: Structural, item, and test generalizability of the Psychopathy Checklist-Revised (PCL-R) in Caucasian and African American Participants, Psychological Assessments, 13(4), 531-542.

17.

Dimitrov, D. M. (2010). Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 43(2), 121-149.

18.

Diniz, A., Pocinho, M. D., & Almeida, L. S. (2011). Cognitive abilities, sociocultural background and academic achievement. Psicothema, 695-700.

19.

Dolan, C. V. (1994). Factor analysis of variables with 2, 3, 5 and 7 response categories: A comparison of categorical variable estimators using simulated data. British Journal of Mathematical and Statistical Psychology, 47(2), 309-326.

20.

Edwards, M. C., Wirth, R. J., Houts, C. R., & Xi, N. (2012). Categorical data in the structural equation modeling framework. In R. Hoyle (Eds.), Handbook of structural equation modeling (pp. 195-208). New York, NY: The Guilford Press.

21.

Elosua, P. (2011). Subjective values of quality of life dimensions in elderly people. A SEM preference model approach. Social indicators research, 104(3), 427-437.

22.

Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological methods, 9(4), 466-491.

23.

Forbush, K. T., Wildes, J. E., Pollack, L. O., Dunbar, D., Luo, J., Patterson, K., ... &Watson, D. (2013). Development and validation of the Eating Pathology Symptoms Inventory (EPSI). Psychological Assessment, 25(3), 859-878.

24.

Forero, C. G., Maydeu-Olivares, A., &Gallardo-Pujol, D. (2009). Factor analysis with ordinal indicators: A Monte Carlo study comparing DWLS and ULS estimation. Structural Equation Modeling, 16(4), 625-641.

25.

French, B. F., & Finch, W. H. (2006). Confirmatory factor analytic procedures for the determination of measurement invariance. Structural Equation Modeling, 13(3), 378-402.

26.

Glanville, J. L., & Wildhagen, T. (2007). The measurement of school engagement: Assessing dimensionality and measurement invariance across race and ethnicity. Educational and Psychological Measurement, 67(6), 1019-1041.

27.

Glockner-Rist, A., & Hoijtink, H. (2003). The best of both worlds: Factor analysis of dichotomous data using item response theory and structural equation modeling. Structural Equation Modeling, 10(4), 544-565.

28.

Hempel, L. M., & Bartkowski, J. P. (2008). Scripture, sin and salvation: Theological conservatism reconsidered. Social Forces, 86(4), 1647-1674.

29.

Hoe, M, Park, B.-S., & Bae, S.-W. (2015). Testing measurement invariance of the 11-tiem Korean version CES-D scale. Meantal Health and Social Work, 43(2), 313-339.

30.

Hutchinson, S. R., & Olmos, A. (1998). Behavior of descriptive fit indexes in confirmatory factor analysis using ordered categorical data. Structural Equation Modeling, 5(4), 344-364.

31.

Johnson, D. R., & Creech, J. C. (1983). Ordinal measures in multiple indicator models: A simulation study of categorization error. American Sociological Review, 48(3) 398-407.

32.

Kamata, A., & Bauer, D. J. (2008). A note on the relation between factor analytic and item response theory models. Structural Equation Modeling, 15(1), 136-153.

33.

Katsikatsou, M., Moustaki, I., Yang-Wallentin, F., & Jöreskog, K. G. (2012). Pairwise likelihood estimation for factor analysis models with ordinal data. Computational Statistics & Data Analysis, 56(12), 4243-4258.

34.

Kim, E. S., Cao, C., Wang, Y., & Nguyen, D. T. (2017). Measurement invariance testing with many groups: a comparison of five approaches. Structural Equation Modeling, 24(4), 524-544.

35.

Kim, E. S., & Yoon, M. (2011). Testing measurement invariance: A comparison of multiple-group categorical CFA and IRT. Structural Equation Modeling, 18(2), 212-228.

36.

Kline, R. B. (2016). Principles and practice of structural equation modeling. New York, NY:Guilford publications.

37.

Kim, S.-Y. (2016). Fundamentals and extensions of structural equation modeling. Seoul: Hakjisa

38.

Koğar, H., & Koğar, E. Y. (2015). Comparison of different estimation methods for categorical and ordinal data in confirmatory factor analysis. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 6(2) 351-364.

39.

Lawrence, J. W., Rosenberg, L., Rimmer, R. B., Thombs, B. D., & Fauerbach, J. A. (2010). Perceived stigmatization and social comfort:Validating the constructs and their measurement among pediatric burn survivors. Rehabilitation Psychology, 55(4), 360-371.

40.

Lee, J. & Yoo, J. P. (2015). Influence of body shape perception on self-esteem among normal-weight middle school students in South Korea: The medicating effect of body shape satisfaction and gender differences. Studies on Korean Youth, 26(4), 267-297.

41.

Lee, P., Chun, S., & Lee, S. (2014). Differential use of middle catetory “?” in job descriptive index between Korean and American samples:Application of mixed-model item response theory. Korean Journal of Psychology: General, 33(3), 647-669.

42.

Lee, S., Lee, C., Lee, H., & Yeo, S. (2012). Invariance of conceptual structure and psychometric properties in Canadian problem gambling index. Korean Journal of Psychology:General, 31(1), 1-26.

43.

Lee, S., Youn, C., Lee, M. M., & Jung, S. (2016). Exploratory factor analysis: How has it changed? Korean Journal of Psychology: General, 35(1), 217-255.

44.

Lee, S. Y., Poon, W. Y., & Bentler, P. M. (1992). Structural equation models with continuous and polytomous variables. Psychometrika, 57(1), 89-105.

45.

Looij-Jansen, P. M., Goedhart, A. W., de Wilde, E. J., & Treffers, P. D. (2011). Confirmatory factor analysis and factorial invariance analysis of the adolescent self‐report Strengths and Difficulties Questionnaire: How important are method effects and minor factors?. British Journal of Clinical Psychology, 50(2), 127-144.

46.

Lopez Rivas, G. E., Stark, S., & Chernyshenko, O. S. (2009). The effects of referent item parameters on differential item functioning detection using the free baseline likelihood ratio test. Applied Psychological Measurement, 33(4), 251-265.

47.

Little, T. D. (1997). Mean and covariance structures (MACS) analyses of cross-cultural data: Practical and theoretical issues. Multivariate Behavioral Research, 32(1), 53-76.

48.

Marsh, H. W. (2007). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. In Perry, R. P. and Smart, J. C. (Eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective (pp. 319-383). Springer, Dordrecht.

49.

Mathyssek, C. M., Olino, T. M., Hartman, C. A., Ormel, J., Verhulst, F. C., & Van Oort, F. V. (2013). Does the Revised Child Anxiety and Depression Scale (RCADS) measure anxiety symptoms consistently across adolescence? The TRAILS study. International Journal of Methods in Psychiatric Research, 22(1), 27-35.

50.

Maydeu-Olivares, A. (2001). Multidimensional item response theory modeling of binary data: Large sample properties of NOHARM estimates. Journal of Educational and Behavioral Statistics, 26(1), 51-71.

51.

Maydeu-Olivares, A., & Cai, L. (2006). A cautionary note on using G2(dif) to assess relative model fit in categorical data analysis. Multivariate Behavioral Research, 41(1), 55-64.

52.

McKelvey, R. D., & Zavoina, W. (1975). A statistical model for the analysis of ordinal level dependent variables. Journal of Mathematical Sociology, 4(1), 103-120.

53.

Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93(3), 568-592.

54.

Mellenbergh, G. J. (1989). Item bias and item response theory. International journal of educational research, 13(2), 127-143.

55.

Meredith, W. (1964). Notes on factorial invariance. Psychometrika, 29(2), 177-185.

56.

Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525-543.

57.

Meuleman, B., & Billiet, J. (2012). Measuring attitudes toward immigration in Europe: The cross-cultural validity of the ESS immigration scales. Research & Methods, 21(1), 5-29.

58.

Millsap, R. E., & Kim, H. (2018). Factorial invariance across multiple populations in discrete and continuous data. In P. Irwing, T. Booth, D. J. Hughes (Eds.), The Wiley Handbook of Psychometric Testing: A Multidisciplinary Reference on Survey, Scale and Test Development (pp. 847-884), Hoboken, NJ, US: Wiley.

59.

Millsap, R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39(3), 479-515.

60.

Moor, M. H., Distel, M. A., Trull, T. J., &Boomsma, D. I. (2009). Assessment of borderline personality features in population samples: Is the Personality Assessment Inventory-Borderline Features scale measurement invariant across sex and age?. Psychological Assessment, 21(1), 125-130.

61.

Morin, A. J., Tran, A., & Caci, H. (2016). Factorial validity of the ADHD Adult Symptom Rating Scale in a French community sample: results from the ChiP-ARD study. Journal of Attention Disorders, 20(6), 530-541.

62.

Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49(1), 115-132.

63.

Muthén, B. (2006). Model fit indices for two-part growth model. Retrived from http://www.statmodel.com/discussion/messages/14/1118.html?1352499941

64.

Muthén, B. (2009). DIFFTEST related. Retrived from http://www.statmodel.com/discussion/messages/11/621.html?1540568737

65.

Muthén, B. (2011). Multiple group analysis with categorical variables. Retrived from http://www.statmodel.com/discussion/messages/23/25.html?1553558399

66.

Muthén, B., & Asparouhov, T. (2002). Latent variable analysis with categorical outcomes:Multiple-group and growth modeling in Mplus. Mplus web notes, 4(5), 1-22.

67.

Muthén, B. O., du Toit, S. H. C., & Spisic, D. (1997). Robust interference using weighted least squares and quadratic estimating equations in the latent variable modeling with categorical and continuous outcomes. Unpublished manuscript, University of California, Los Angeles, USA.

68.

Muthen, B., & Kaplan, D. (1992). A comparison of some methodologies for the factor analysis of non‐normal Likert variables: A note on the size of the model. British Journal of Mathematical and Statistical Psychology, 45(1), 19-30.

69.

Muthén, L., & Muthén, B. (1998-2019). Mplus (Version 7). Los Angeles, CA: Muthén &Muthén.

70.

Nussbeck, F. W., Eid, M., & Lischetzke, T. (2006). Analyzing multitrait-multimethod data with sturctural equation models for ordinal variables applying the WLSMV estimator:What sample size is needed for valid results? British Journal of Mathematical and Statistical Psychology, 59(1), 195-213.

71.

Potthast, M. J. (1993). Confirmatory factor analysis of ordered categorical variables with large models. British Journal of Mathematical and Statistical Psychology, 46(2), 273-286.

72.

Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114(3), 552-566.

73.

Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354-373.

74.

Rutkowski, L., & Svetina, D. (2014). Assessing the hypothesis of measurement invariance in the context of large-scale international surveys. Educational and Psychological Measurement, 74(1), 31-57.

75.

Rutkowski, L., & Svetina, D. (2017). Measurement invariance in international surveys: Categorical indicators and fit measure performance. Applied Measurement in Education, 30(1), 39-51.

76.

Sass, D. A., & Schmitt, T. A. (2013). Testing measurement and structural invariance:Implications for practice. In T. Teo (Eds.)Handbook of quantitative methods for educational research (pp. 315-345). Rotterdam: Sense Publishers.

77.

Sass, D. A., Schmitt, T. A., & Marsh, H. W. (2014). Evaluating model fit with ordered categorical data within a measurement invariance framework: A comparison of estimators. Structural Equation Modeling, 21(2), 167-180.

78.

Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66(4), 507-514.

79.

Satterthwaite, F. E. (1941). Synthesis of variance. Psychometrika, 6(5), 309-316.

80.

Schlotz, W., Yim, I. S., Zoccola, P. M., Jansen, L., & Schulz, P. (2011). The perceived stress reactivity scale: Measurement invariance, stability, and validity in three countries. Psychological assessment, 23(1), 80-94.

81.

Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of educational research, 99(6), 323-338.

82.

Selig, J. P., Card, N. A., & Little, T. D. (2008). Latent variable structural equation modeling in cross-cultural research: Multigroup and multilevel approaches. In F. J. R. van de Vijver, & D. A. van Hemert, Y. H. Poortinga (Eds.), Multilevel analysis of individuals and cultures(pp. 93-119), New York, NY: Taylor & Francis.

83.

Seo, D. & Lee, S. (2017). Item response theory approach using item factor anlaysis in the development of psycholgocial test. A packet for the Korean Psychological Association, 137.

84.

Stark, S., Chernyshenko, O. S., Drasgow, F., &Williams, B. A. (2006). Examining assumptions about item responding in personality assessment: Should ideal point methods be considered for scale development and scoring?. Journal of Applied Psychology, 91(1), 25-39.

85.

Steenkamp, J. B. E., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of consumer research, 25(1), 78-90.

86.

Temme, D. (2006). Assessing measurement invariance of ordinal indicators in cross-national research. In S. Diehl, R. Terlutter (Eds.), International advertising and communication: Current insights and empirical findings (pp. 455-472), Springer Science &Business Media.

87.

Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4-70.

88.

Wang, W. C., & Yeh, Y. L. (2003). Effects of anchor item methods on differential item functioning detection with the likelihood ratio test. Applied Psychological Measurement, 27(6), 479-498.

89.

Widaman, K. F. (2000). Testing cross-group and cross-time constraints on parameters using the general linear model. In T. D. Little, K.U. Schnabel, & J. Baumert (Eds.), Modeling longitudinal and multilevel data: Practical issues, applied approaches, and specific examples (pp. 163-185, 269-281). Mahwah, NJ, US:Lawrence Erlbaum Associates Publishers.

90.

Widaman, K. F., & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In K. J. Bryant, M. Windle, & S. G. West (Eds.), The science of prevention:Methodological advances from alchol and substance abuse research (pp. 281-324), Washington, DC, US: American Psychologocal Association.

91.

Wu, H., & Estabrook, R. (2016). Identification of confirmatory factor analysis models of different levels of invariance for ordered categorical outcomes. Psychometrika, 81(4), 1014-1045.

92.

Xing, C., & Hall, J. A. (2015). Confirmatory factor analysis and measurement invariance testing with ordinal data: An application in revising the flirting styles inventory. Communication Methods and Measures, 9(3), 123-151.

93.

Yang, Y., & Green, S. B. (2010). A note on structural equation modeling estimates of reliability. Structural Equation Modeling, 17(1), 66-81.

94.

Yang-Wallentin, F., Jöreskog, K. G., & Luo, H. (2010). Confirmatory factor analysis of ordinal variables with misspecified models. Structural Equation Modeling, 17(3), 392-423.

95.

Yoon, M., & Millsap, R. E. (2007). Detecting violations of factorial invariance using data-based specification searches: A Monte Carlo study. Structural Equation Modeling, 14(3), 435-463.

96.

Liu, Y., Millsap, R. E., West, S. G., Tein, J.-Y., Tanaka, R., & Grimm, K. J. (2017). Testing measurement invariance in longitudinal data with ordered-categorical measures. Psychological Methods, 22(3), 486-506.

97.

Zimprich, D., Allemand, M., & Dellenbach, M. (2009). Openness to experience, fluid intelligence, and crystallized intelligence in middle-aged and old adults. Journal of Research in Personality, 43(3), 444-454.

Korean Journal of Psychology: General