open access
메뉴
open access
메뉴
ISSN : 1229-0696
This study demonstrates how a multigroup analysis approach is used in the analysis of multilevel data to judge if a referent-shift consensus model is needed to measure a compositional property. A compositional property in multilevel context means that the forms of emergence from individual levels to group levels are isomorphic as individuals interact, communicate perspectives, and iteratively construct a common interpretation, so that all individuals in the collective are similar. The measurement principle for conceptualization of multilevel compositional properties is to use a referent-shift consensus model proposed by Chan(1998). However, if the researcher wants to use the samel construct in individual levels as well as in group levels, she needs to administer the same items to the same individuals again with a change of reference from “group” to “ individual”. It sounds bothersome and creates difficulties in reality. For that reason, researchers often collect data from individuals using self-referenced items, aggregate, and then use the aggregate scores as measures of group level variables. However in these cases, measurement invariance is tacitly assumed across the individual and group levels. We pointed out the problems of this unjustified assumption in analyses of multilevel data, and presented an analytic procedure to test the assumption using multigroup analysis framework. In sum, if measurement invariance across levels is established, researchers can use either a self-referent or a referent-shift data at individual levels and aggregate data at group levels without dual measurement. Moreover, in such a case, using a referent-shift data(‘we’ data) is more appropriate in light of construct validity because of its higher possibility to reveal group effects. If the measurement invariance across levels is not supported, researchers should collect data separately for individual level variables with a self-reference items and for group level variables with group-referenced items.
김아영, 차정은, 이채희, 서애리, & 최기연 (2004). 학교급간 학업적 자기조절척도의 구인동등성 검증 및 잠재평균분석. 교육심리연구, 18(2), 227-244.
이순묵. (1993). 중급 LISREL. 한국심리학회 동계연수회 자료집.
이순묵. (2010). 역량과 역량관련 프로그램의 타당화를 위한 제안. 한국 심리학회지: 산업 및 조직, 23(3), 551-573.
이순묵. (2014). 구조방정식 모형의 일반화와 차별화: 다집단 분석. 미발간
이순묵, 김한조. (2011). 구조방정식 모형의 일반화 또는 집단차 연구를 위한 다집단 분석의 관행과 문제점. 사회과학(성균관대), 43(1), 63-112.
이순묵, 윤수철, 차정은, 김종남, & 여성칠. (2012). 한국판 CPGI 와 원본척도 (CPGI)간 측정동등성 및 점수연계 가능성. 한국심리학회지: 임상, 31(2), 401-425.
이회영. (2012). 심리적 자본과 조직공정성이 직무만족과 조직몰입에 미치는 영향. 성균관대학교 석사학위 논문.
이회영, 김종규, & 이순묵 (2013). 포스터 발표: 3 분과 산업 및 조직; 심리적 자본과 조직공정성이 직무만족과 조직몰입에 미치는 영향: 다수준적 (multi-level) 고찰. 한국심리학회 연차 학술발표논문집, 2013(1), 314.
최은하 (2012). 팀 유연성에 대한 타당도 분석: 다수준적 관점의 적용. 성균관대학교 석사학위 논문.
최은하, & 구병모 (2013). 포스터 발표: 3 분과 산업 및 조직; 상향적으로 정의되는 팀 유연성의 타당화.
한정원, 이경수, 박찬신, & 손영우 (2009). 조종사의 안전행동을 예측하는 조직의 안전문화와 개인의 안전태도 및 안전동기 간의 관계. 한국심리학회지: 산업 및 조직, 22 (1), 109-129.
한태영, & 김원형 (2006). 권한위임과 조직공정성이 직무효과성에 미치는 영향에 대한 다수준적(multilevel) 고찰. 인사조직연구, 14, 183-216.
홍세희, 황매향, & 이은설 (2005). 청소년용 여성 진로장벽 척도의 잠재평균분석. 교육심리연구, 19(4), 1159-1177.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological bulletin, 107(2), 238.
Bentler, P. M. (1995). EQS structural equations program manual. Multivariate Software.
Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological bulletin, 88(3), 588.
Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In K. J. Klein & Steve W. J. Kozlowski(Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions(pp. 349-381). San Francisco, CA, US: Jossey-Bass, xxix, 605 pp.
Bovaird, J. A., & Koziol, N. A. (2012). Measurement Models for Ordered-Categorical Indicators. In Hoyle, R. H. (Ed.) Handbook of Structural Equation Modeling. NY: Guilford Press.
Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological bulletin, 105(3), 456.
Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied psychology, 83, 234-246.
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464-504.
Chen, G., Bliese, P. D., & Mathieu, J. E. (2005). Conceptual framework and statistical procedures for delineating and testing multilevel theories of homology. Organizational Research Methods, 8(4), 375-409.
Chen, G., Mathieu, J. E., & Bliese, P. D. (2004). A framework for conducting multilevel construct validation. Research in Multi Level Issues, 3, 273-303.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural equation modeling, 9(2), 233-255.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
Cronbach, L. J. (1976). Research on classrooms and schools: Formulation of questions, design and analysis. Occasional Paper of the Stanford Evaluation Consortium, Stanford University.
DeShon, R. P., Kozlowski, S. W., Schmidt, A. M., Milner, K. R., & Wiechmann, D. (2004). A multiple-goal, multilevel model of feedback effects on the regulation of individual and team performance. Journal of Applied Psychology, 89(6), 1035.
Dyer, N. G., Hanges, P. J., & Hall, R. J. (2005). Applying multilevel confirmatory factor analysis techniques to the study of leadership. The leadership quarterly, 16(1), 149-167.
Goldstein, I. L. (1993). Training in organizations: Needs assessment, development, and evaluation. Thomson Brooks/Cole Publishing Co.
Griffin. M. A., & Mason, C. M. (2002). Grouptask Satisfaction : Applying the Construct of Job Satisfaction to Groups. Small Group Research, 33, 271-312.
Guzzo, R. A., Yost, P. R., Campbell, R. J., & Shea, G. P. (1993). Potency in groups: Articulating a construct. British journal of social psychology, 32(1), 87-106.
Heck, R. H. (2001). Multilevel modeling with SEM. New developments and techniques in structural equation modeling, 89-127.
James, L. R. (1982). Aggregation bias in estimates of perceptual agreement. Journal of Applied Psychology, 67, 219-229.
James, L. R., Demaree, R. G., & Hater, J. J. (1980). A statistical rationale for relating situational variables and individual differences. Organizational Behavior and Human Performance, 25(3), 354-364.
James, L. R., Demaree, R. G., &Wolf, G. (1984). Estimating within-group interrater reliability with and without response bias. Journal of applied psychology, 69(1), 85.
James, L. R., & Jones, A. P. (1974). Organizational climate: A review of theory and research. Psychological bulletin, 81(12), 1096.
Jones, A. P., & James, L. R. (1979). Psychological climate: Dimensions and relationships of individual and aggregated work environment perceptions. Organizational behavior and human performance, 23(2), 201-250.
Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations, Psychometrika, 36(4). 409-426.
Kenny, D. A. (2010). Fit measures. http://davidakenny.net/cm/fit.htm.
Kozlowski, S.W., & Hattrup, K. 1992. A disagreement about within-group agreement: Disentangling issues of consistency versus consensus. Journal of Applied Psychology, 77: 161-167.
Kozlowski, S. W. J., Gully, S. M., Nason, E. R., Ford, J. K., Smith, E. M., Smith, M. R., & Futch, C. J. (1994). A composition theory of team development: Levels, content, process, and learning outcomes. In JE Mathieu (Chair), Developmental views of team process and performance. Symposium conducted at the ninth annual conference of the Society for Industrial and Organizational Psychology, Nashville, TN.
Kozlowski, S. W., & Hults, B. M. (1987). An exploration of climates for technical updating and performance. Personnel Psychology, 40(3), 539-563.
Kozlowski, S. W., & Klein, K. J. (2000). A Multilevel approach to theory and research in organizations: Contextual, Temporal, and Emergent Processes. In K. J. Klein & S. W. J. Kozloeski (Eds.), Multilevel Theory, Research, and Methods in Organizations. San Francisco: Jossey-Bass.
Kozlowski, S. W., & Salas, E. (1997). A multilevel organizational systems approach for the implementation and transfer of training. Improving training effectiveness in work organizations, 247, 287.
Kozlowski, S.W.J. (1998, March). Extending and elaborating models of emergent phenomena. Presentation at MESO Organization Studies Group, Arizona State University, Tempe.
Kozlowski, S.W.J. (1999, April). A typology of emergence: Theoretical mechanisms understanding bottom-up phenomena in organizations. In F. P. Morgeson & D.A. Hofmann (Chairs), New perspectives on higher-level phenomenon in industrial/organizational psychology. Symposium conducted at the fourteenth annual conference of the Society for Industrial and Organizational Psychology, Atlanta, GA.
LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11, 815-852. doi: 10.1177/10944281 06296642
Lee, B. H. (2003). An Empirical Study of Organizational Commitment: A Multi-Level Approach. Journal of Behavioral and Applied Management, 4(3), 176-188.
Maas, C. J., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology, 1(3), 86-92.
Marsh, H. W., Lüdtke, O., Robitzsch, A., Trautwein, U., Asparouhov, T., Muthén, B., & Nagengast, B. (2009). Doubly-latent models of school contextual effects: Integrating multilevel and structural equation approaches to control measurement and sampling error. Multivariate Behavioral Research, 44(6), 764-802.
McGehee, W., & Thayer, P. W. (1961). Training in business and industry.
Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525-543.
Moore, D. S. (2010). The basic practice of statistics. (5th Ed.). NY: Freeman & Company.
Moorman, R. H. (1991). Relationship between organizational justice and organizational citizenship behaviors: do fairness perceptions influence employee citizenship?. Journal of applied psychology, 76(6), 845.
Morgeson, F. P., & Hofmann, D. A. (1999). The structure and function of collective constructs: Implications for multilevel research and theory development. Academy of Management Review, 24(2), 249-265.
Muthén, B. O. (1991). Multilevel factor analysis of class and student achievement components. Journal of Educational Measurement, 28(4), 338- 354.
Muthen, B. O. (1994). Multilevel covariance structure analysis. Sociological methods & research, 22(3), 376-398.
Muthen, L. K., & Muthen, B. O. (1998). Mplus [computer software]. Los Angeles, CA: Muthén & Muthén.
Muthén, B. O., & Satorra, A. (1989). Multilevel aspects of varying parameters in structural models. Multilevel analysis of educational data, 87-99.
Muthén, B. O., & Satorra, A. (1995). Technical aspects of Muthén's LISCOMP approach to estimation of latent variable relations with a comprehensive measurement model. Psychometrika, 60(4), 489-503.
Myers, N. D., Feltz, D. L., & Short, S. E. (2004). Collective Efficacy and Team Performance: A Longitudinal Study of Collegiate Football Teams. Group Dynamics: Theory, Research, and Practice, 8(2), 126.
Myers, N. D., Payment, C. A., & Feltz, D. L. (2004). Reciprocal Relationships Between Collective Efficacy and Team Performance in Women's Ice Hockey. Group Dynamics: Theory, Research, and Practice, 8(3), 182.
Niehoff, B. P., & Moorman, R. H. (1993). Justice as a mediator of the relationship between methods of monitoring and organizational citizenship behavior. Academy of Management journal, 36(3), 527-556.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). Sage.
Schmitt, N., & Kuljanin, G. (2008). Measurement invariance: Review of practice and implications. Human Resource Management Review, 18(4), 210-222.
Schweig, J. (2013). Testing the assumption of cross-level measurement invariance in multilevel models: Evidence from school and classroom environment surveys (CRESST Report 829). Los Angeles, CA: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
Sirotnik, K. A. (1980). Psychometric implications of the unit‐of‐analysis problem (with examples from the measurement of organizational climate). Journal of Educational Measurement, 17(4), 245-282.
Snijders T. A. B., & Bosker RJ (1999). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. Sage Publications, London.
Sörbom, D. (1974). A general method for studying differences in factor means and factor structure between groups. British Journal of Mathematical and Statistical Psychology, 27(2), 229-239.
Stapleton, L. M. (2006). Using multilevel structural equation modeling techniques with complex sample data. In Hancock, G. R., & Mueller, R. O(Eds.) , Structural equation modeling: A second course. pp.345-383. Greenwich, CON: Information age publishing.
Steiger, J. H., & Lind, J. C. (1980, May). Statistically based tests for the number of common factors. In annual meeting of the Psychometric Society, Iowa City, IA (Vol. 758, pp. 424-453).
Tay, L., Woo, S. E., & Vermunt, J. K. (2014). A Conceptual and Methodological Framework for Psychometric Isomorphism Validation of Multilevel Construct Measures. Organizational Research Methods, 17(1), 77-106.
Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational research methods, 3(1), 4-70.
Yoon, M., & Millsap, R.E. ( 2007). Detecting violations of factorial invariance using data- based specification searches: A Monte Carlo study. Structural Equation Modeling, 14. 435-463.
Zyphur, M. J., Kaplan, S. A., & Christian, M. S. (2008). Assumptions of cross-level measurement and structural invariance in the analysis of multilevel data: Problems and solutions. Group Dynamics: Theory, Research, and Practice, 12(2), 127.