바로가기메뉴

본문 바로가기 주메뉴 바로가기

Korean Journal of Psychology: General

Differential use of middle category “?” in Job Descriptive Index Between Korean and American Samples: Application of mixed-model item response theory

Korean Journal of Psychology: General / Korean Journal of Psychology: General, (P)1229-067X; (E)2734-1127
2014, v.33 no.3, pp.647-669
(University of South Florida)
(University of South Florida)

Abstract

The present study investigated differential use of the middle category “?” option between Korean and American workers using the 2009 version Job Descriptive Index(JDI). For this purpose, a multi-group categorical confirmatory factor analysis and a mixed-model item response theory(MM-IRT) analysis were conducted across a Korean worker sample and American normative sample (N=932 for each). The multi-group analysis result supported configural and metric invariance across the groups. However, the threshold structure was not invariant across the groups. The MM-IRT analysis identified three latent subgroups within both Korean and American samples. Overall, the Korean group has larger subgroup class size favoring the “?” option than the American group. An inspection of item parameters revealed the threshold locations of the mixed partial credit model were frequently disordered for the American group, but not for the Korean group. The implications of the results and future research issues were discussed.

keywords
Likert item, middle category response option, measurement invariance, mixed-model item response theory, Job Descriptive Index, 평정척도 문항, 중간 응답범주, 측정동일성, 혼합문항반응이론, 직무기술척도

Reference

1.

Andrich, D., Jong, J., & Sheidan, B. E. (1997). Diagnostic opportunities with the Rasch model for ordered response categories. In J. Rost & R. Langeherine (Eds.) Applications of latent trait and latent class models in the social sciences (pp.58-68). Munster, Germany: Waxman Verlag.

2.

Andrich, D., & Schoibroeck, L. V. (1989). The general health questionnaire: a psychometric analysis using latent trait theory. Psychological Medicine, 19, 469-485.

3.

Bowling, N. A., Hendricks, E. A. & Wagner, S. H. (2008). Positive and negative affectivity and facet satisfaction: A meta-analysis. Journal of Business and Psychology, 23, 115-125.

4.

Bozdogan, H. (1987). Model selection and Akaike’s information criterion (AIC): The general theory and its analytical extensions. Psychometrika, 52, 345-370.

5.

Brislin, R. W. (1970). Back-translation for cross-cultural research. Journal of Cross-Cultural Psychology, 1, 185-216.

6.

Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York, NY: Guildford Press.

7.

Brown, A., & Maydeu-Olivares, A. (2013). How IRT can solve problems of Ipsative data in forced-choice questionnaires. Psychological Methods, 18, 36-52.

8.

Carter, N. T., Dalal, D. K., Lake, C. J., Lin, B. C., & Zickar, M. J. (2011). Using mixed-model item response theory to analyze organizational survey responses: An illustration using the Job Descriptive Index. Organizational Research Methods, 14, 116-146.

9.

Cattell, R. B., & Cattell, H. P. (1995). Personality structure and the new fifth edition of the 16PF. Educational and Psychological Measurement, 55, 926-937.

10.

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, 464-504.

11.

Cheung, K. C., & Mooi, L. C. (1994). A comparison between the rating scale model and dual scaling for Likert scales. Applied psychological Measurement, 18, 1-13.

12.

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255.

13.

Dubois, B., & Burns, J. A. (1975). An analysis of the meaning of the question mark response category in attitude scales. Educational & Psychological Measurement, 35, 869-884.

14.

Eid, M., & Rauber, M. (2000). Detecting measurement invariance in organizational surveys. European Journal of Psychological Assessment, 16, 20-30.

15.

Gonzalez-Roma, V. & Espejo, B. (2003). Testing the middle response categories “Not sure”, “In between” and “?” in polytomous items. Psicothema, 15, 278-284.

16.

Hernández, A., Drasgow, F., & González-Romá, V. (2004). Investigating the functioning of a middle category by means of a mixed-measurement model. Journal of Applied Psychology, 89, 687-699.

17.

Herna'ndez, A., Espejo B., & Gonza´lez-Roma, V. (2006). The functioning of central categories middle level and sometimes in graded response scales: does the label matter? Psicothema, 18, 300-306.

18.

Hu, L. T., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424-453.

19.

Hui, C. H., & Triandis, H. C. (1985). Measurement in cross-cultural psychology: A review and comparison of strategies. Journal of Cross-Cultural Psychology, 16, 131-152.

20.

Kieruj, N. D., & Moors, G. (2010). Variations in response style behavior by response scale format in attitude research. Journal of Public Opinion Research, 22, 320-342.

21.

Kinicki, A. J., McKee-Ryan, F. M., Schriesheim, C. A., & Carson, K. P. (2002). Assessing the construct validity of the Job Descriptive Index: A review and meta-analysis. Journal of Applied Psychology, 87, 14-32.

22.

Kolen, M. J., & Brennan, R. L. (2004). Test equating, scaling, and linking: Methods and practices (2nd ed.). New York: Springer-Verlag.

23.

Kulas, J. T. & Stachowski, A. A. (2009). Middle category endorsement in odd-numbered Likert response scales: Associated item characteristics, cognitive demands, and preferred meanings. Journal of Research in Personality, 43, 489-493.

24.

Lake, C. J., Gopalkrishnan, P., Sliter, M. T., & Withrow, S. (2010). The Job Descriptive Index: Newly updated and available for download. The Industrial-Organizational Psychologist, 48, 47-49.

25.

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale NJ: Erlbaum.

26.

Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.

27.

Maij-de Meij, A. M., Kelderman, H., & van der Flier, H. (2008). Fitting a mixture item response theory model to personality questionnaire data: Characterizing latent classes and investigating possibilities for improving prediction. Applied Psychological Measurement, 32, 611-631.

28.

Millsap R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39, 479-515.

29.

Moors, G. (2008). Exploring the effect of a middle response category on response style in attitude measurement. Quality and Quantity, 42, 779- 794.

30.

Muthén, L. K., & Muthén, B. O. (2010). Mplus user’s guide (6th ed.). Los Angeles, CA: Muthén & Muthén.

31.

Raju, N. S., Laffitte, L. J., & Byrne, B. M. (2002). Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory. Journal of Applied Psychology, 87, 517-529.

32.

Riordan, C. M., & Vandenberg, R. J. (1994). A central question in cross-cultural research: Do employees of different cultures interpret work-related measures in an equivalent manner? Journal of Management, 20, 643-671.

33.

Rost, J. (1997). Logistic mixture models. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 449-463). New York: Springer.

34.

Smith, P. C., & Kendall, L., & Hulin, C. I. (1969). The measurement of satisfaction in work and retirement: A strategy for the study of attitudes. Chicago: Rand McNally.

35.

Tak, J., & Downey, R. G. (1991). Assessing construct validity of R-JDI in Korea. 한국심리학회지: 산업 및 조직, 4, 87-91.

36.

Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-69.

37.

von Davier, M. (2001). WINMIRA2001: Windows mixed Rasch model analysis [Computer software and User manual]. Kiel, the Netherlands: Institute for Science Education.

38.

von Davier, M., Rost, J., & Carstensen, C. H. (2007). Introduction: Extending the Rasch model. In M. von Davier & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications. New York: Springer.

39.

Wang, M., & Russell, S. S. (2005). Measurement equivalence of the Job Descriptive Index across Chinese and American workers: Results from confirmatory factor analysis and item response theory. Educational and Psychological Measurement, 65, 709-732.

40.

Wilde, G. J. S. (1970). Neurotische labiliteit gemeten volgens de vragen - lijstmethode. Amsterdam: Van Rossem.

41.

Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current and future directions. Psychological Methods, 12, 58-79

42.

Zickar, M. J., Gibby, R. E., & Robie, C. (2004). Uncovering faking samples in applicant, incumbent, and experimental data sets: An application of mixed-model item response theory. Organizational Research Methods, 7, 68-190.

Korean Journal of Psychology: General