Differential use of middle category “?” in Job Descriptive Index Between Korean and American Samples: Application of mixed-model item response theory

본 연구는 한국과 미국의 응답자들이 평정척도 상의 중간보기 “?”를 서로 다르게 사용하는지에 대한 비교 문화적 검증을 실시하였다. 이를 위해 직무기술척도(Job Descriptive Index)에 대한 한국과 미국 집단(각각 932명)의 응답을 다집단 범주적 확인적 요인분석(Multi-Group Categorical Confirmatory Factor Analysis)과 혼합문항반응이론(Mixed-Model Item Response Theory)으로 분석하였다. 다집단 분석결과, 두 집단 간에 측정틀 동일성과 측정단위 동일성은 성립하였으나, 임계치 모수의 구조에서는 차이가 있는 것으로 나타났다. 혼합문항반응이론을 실시한 결과, 한국과 미국집단 모두에서 응답범주에 대한 반응양상이 다른 세 개의 하위 잠재집단이 존재하였다. 이 중 “?” 응답범주를 많이 선택하는 “?” 선호집단에 대한 보다 자세한 분석 결과, 미국 표본에서는 “?” 선호 집단의 비중이 매우 적고, 응답범주의 순차성 가정이 지지되지 않는 것으로 나타났다. 반면, 한국 표본에서는 “?” 선호집단이 다른 하위 잠재집단과 유사한 비중을 차지할 뿐만 아니라, “?”을 비교적 일정한 간격을 지니는 순차적 평정척도의 중간보기로 인식하는 것으로 나타났다. 본 연구의 결과를 바탕으로 중간보기 “?”의 사용에 대한 적용적 시사점 및 추후 연구 과제를 논의하였다.

keywords: Likert item, middle category response option, measurement invariance, mixed-model item response theory, Job Descriptive Index, 평정척도 문항, 중간 응답범주, 측정동일성, 혼합문항반응이론, 직무기술척도

Abstract

The present study investigated differential use of the middle category “?” option between Korean and American workers using the 2009 version Job Descriptive Index(JDI). For this purpose, a multi-group categorical confirmatory factor analysis and a mixed-model item response theory(MM-IRT) analysis were conducted across a Korean worker sample and American normative sample (N=932 for each). The multi-group analysis result supported configural and metric invariance across the groups. However, the threshold structure was not invariant across the groups. The MM-IRT analysis identified three latent subgroups within both Korean and American samples. Overall, the Korean group has larger subgroup class size favoring the “?” option than the American group. An inspection of item parameters revealed the threshold locations of the mixed partial credit model were frequently disordered for the American group, but not for the Korean group. The implications of the results and future research issues were discussed.

keywords: Likert item, middle category response option, measurement invariance, mixed-model item response theory, Job Descriptive Index, 평정척도 문항, 중간 응답범주, 측정동일성, 혼합문항반응이론, 직무기술척도

참고문헌

Andrich, D., Jong, J., & Sheidan, B. E. (1997). Diagnostic opportunities with the Rasch model for ordered response categories. In J. Rost & R. Langeherine (Eds.) Applications of latent trait and latent class models in the social sciences (pp.58-68). Munster, Germany: Waxman Verlag.

Andrich, D., & Schoibroeck, L. V. (1989). The general health questionnaire: a psychometric analysis using latent trait theory. Psychological Medicine, 19, 469-485.

Bowling, N. A., Hendricks, E. A. & Wagner, S. H. (2008). Positive and negative affectivity and facet satisfaction: A meta-analysis. Journal of Business and Psychology, 23, 115-125.

Bozdogan, H. (1987). Model selection and Akaike’s information criterion (AIC): The general theory and its analytical extensions. Psychometrika, 52, 345-370.

Brislin, R. W. (1970). Back-translation for cross-cultural research. Journal of Cross-Cultural Psychology, 1, 185-216.

Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York, NY: Guildford Press.

Brown, A., & Maydeu-Olivares, A. (2013). How IRT can solve problems of Ipsative data in forced-choice questionnaires. Psychological Methods, 18, 36-52.

Carter, N. T., Dalal, D. K., Lake, C. J., Lin, B. C., & Zickar, M. J. (2011). Using mixed-model item response theory to analyze organizational survey responses: An illustration using the Job Descriptive Index. Organizational Research Methods, 14, 116-146.

Cattell, R. B., & Cattell, H. P. (1995). Personality structure and the new fifth edition of the 16PF. Educational and Psychological Measurement, 55, 926-937.

10.

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, 464-504.

11.

Cheung, K. C., & Mooi, L. C. (1994). A comparison between the rating scale model and dual scaling for Likert scales. Applied psychological Measurement, 18, 1-13.

12.

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255.

13.

Dubois, B., & Burns, J. A. (1975). An analysis of the meaning of the question mark response category in attitude scales. Educational & Psychological Measurement, 35, 869-884.

14.

Eid, M., & Rauber, M. (2000). Detecting measurement invariance in organizational surveys. European Journal of Psychological Assessment, 16, 20-30.

15.

Gonzalez-Roma, V. & Espejo, B. (2003). Testing the middle response categories “Not sure”, “In between” and “?” in polytomous items. Psicothema, 15, 278-284.

16.

Hernández, A., Drasgow, F., & González-Romá, V. (2004). Investigating the functioning of a middle category by means of a mixed-measurement model. Journal of Applied Psychology, 89, 687-699.

17.

Herna'ndez, A., Espejo B., & Gonza´lez-Roma, V. (2006). The functioning of central categories middle level and sometimes in graded response scales: does the label matter? Psicothema, 18, 300-306.

18.

Hu, L. T., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424-453.

19.

Hui, C. H., & Triandis, H. C. (1985). Measurement in cross-cultural psychology: A review and comparison of strategies. Journal of Cross-Cultural Psychology, 16, 131-152.

20.

Kieruj, N. D., & Moors, G. (2010). Variations in response style behavior by response scale format in attitude research. Journal of Public Opinion Research, 22, 320-342.

21.

Kinicki, A. J., McKee-Ryan, F. M., Schriesheim, C. A., & Carson, K. P. (2002). Assessing the construct validity of the Job Descriptive Index: A review and meta-analysis. Journal of Applied Psychology, 87, 14-32.

22.

Kolen, M. J., & Brennan, R. L. (2004). Test equating, scaling, and linking: Methods and practices (2nd ed.). New York: Springer-Verlag.

23.

Kulas, J. T. & Stachowski, A. A. (2009). Middle category endorsement in odd-numbered Likert response scales: Associated item characteristics, cognitive demands, and preferred meanings. Journal of Research in Personality, 43, 489-493.

24.

Lake, C. J., Gopalkrishnan, P., Sliter, M. T., & Withrow, S. (2010). The Job Descriptive Index: Newly updated and available for download. The Industrial-Organizational Psychologist, 48, 47-49.

25.

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale NJ: Erlbaum.

26.

Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.

27.

Maij-de Meij, A. M., Kelderman, H., & van der Flier, H. (2008). Fitting a mixture item response theory model to personality questionnaire data: Characterizing latent classes and investigating possibilities for improving prediction. Applied Psychological Measurement, 32, 611-631.

28.

Millsap R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39, 479-515.

29.

Moors, G. (2008). Exploring the effect of a middle response category on response style in attitude measurement. Quality and Quantity, 42, 779- 794.

30.

Muthén, L. K., & Muthén, B. O. (2010). Mplus user’s guide (6th ed.). Los Angeles, CA: Muthén & Muthén.

31.

Raju, N. S., Laffitte, L. J., & Byrne, B. M. (2002). Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory. Journal of Applied Psychology, 87, 517-529.

32.

Riordan, C. M., & Vandenberg, R. J. (1994). A central question in cross-cultural research: Do employees of different cultures interpret work-related measures in an equivalent manner? Journal of Management, 20, 643-671.

33.

Rost, J. (1997). Logistic mixture models. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 449-463). New York: Springer.

34.

Smith, P. C., & Kendall, L., & Hulin, C. I. (1969). The measurement of satisfaction in work and retirement: A strategy for the study of attitudes. Chicago: Rand McNally.

35.

Tak, J., & Downey, R. G. (1991). Assessing construct validity of R-JDI in Korea. 한국심리학회지: 산업 및 조직, 4, 87-91.

36.

Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-69.

37.

von Davier, M. (2001). WINMIRA2001: Windows mixed Rasch model analysis [Computer software and User manual]. Kiel, the Netherlands: Institute for Science Education.

38.

von Davier, M., Rost, J., & Carstensen, C. H. (2007). Introduction: Extending the Rasch model. In M. von Davier & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications. New York: Springer.

39.

Wang, M., & Russell, S. S. (2005). Measurement equivalence of the Job Descriptive Index across Chinese and American workers: Results from confirmatory factor analysis and item response theory. Educational and Psychological Measurement, 65, 709-732.

40.

Wilde, G. J. S. (1970). Neurotische labiliteit gemeten volgens de vragen - lijstmethode. Amsterdam: Van Rossem.

41.

Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current and future directions. Psychological Methods, 12, 58-79

42.

Zickar, M. J., Gibby, R. E., & Robie, C. (2004). Uncovering faking samples in applicant, incumbent, and experimental data sets: An application of mixed-model item response theory. Organizational Research Methods, 7, 68-190.

바로가기메뉴

논문 상세

Vol.33 No.3

중간보기 “?” 의 차별적 사용에 대한 한국과 미국 간 비교 문화적 검증: 혼합 문항반응분석의 적용

Differential use of middle category “?” in Job Descriptive Index Between Korean and American Samples: Application of mixed-model item response theory

초록

Abstract

참고문헌

한국심리학회지: 일반