Development and Validation of the Korean School Engagement Scale for Middle School Students (K-SES-M) Short Form
Abstract
The Korean School Engagement Scale for Middle School Students (K-SES-M) is a 48-item self-report scale, which measures middle school students’ affective, behavioral, social, and academic engagement in school. Although the K-SES-M well reflects how Korean middle school students engage in school and is reliable and valid, there still remain psychometric issues to be resolved in the procedure of item development and quite a number of items restrict its use for both practical and research purposes. This study was conducted to develop a brief, psychometrically sound version of the K-SES-M using Rasch measurement model and to test its reliability and validity. Seven-hundred thirty-four students in Grades 7～9 completed the K-SES-M and criterion tests. The Rasch rating scale model was applied to each of the four school engagement factors. Results showed that a 4-point rating scale was more appropriate for the K-SES-M short form (K-SES-MS) than was the 5-point rating scale of the original K-SES-M. Based on item fit and item difficulty estimates for each of the four factors as well as differential item functioning analyses by gender for each item, 24 items were initially selected. The results from confirmatory factor analyses indicated that the fit of the first-order three-factor model consisting of affective engagement (7 items), behavioral engagement (8 items), and academic engagement (7 items) was reasonable. Thus, a 22-item short form of the K-SES-M was presented to meet the need for a measure that could be used for periodic monitoring of Korean middle school students’ engagement in school. The three factors of school engagement were found to have adequate construct reliability and internal consistency, and the K-SES-MS was found to have strong concurrent evidence. Finally, the use of the K-SES-MS, limitations of this study and suggestions for further research were discussed.
- keywords
- 학교참여, 학교생활참여척도, 단축형, K-SES-MS, Rasch 평정척도모형, school engagement, school engagement scale, short form, K-SES-MS, Rasch rating scale model
Reference
교육부 (2016). 학업중단 학생 현황. 교육부.
김명숙, 설현수 (2007). Rasch 측정모형에 의한 K 비판적 사고성향 검사의 양호도 검증. 교육평가연구, 20(2), 101-124.
김성숙, 박찬옥, 설현수 (2009). Rasch 측정 모형을 활용한 유아 리더십 유형 검사 도구 타당화. 열린유아교육연구, 14(3), 517-556.
김종렬, 이은주 (2012). 초․중학교급별 교실목표구조, 기본심리욕구, 수업참여 간의 구조적 관계 분석. 교육심리연구, 26(3), 817-835.
김종백, 김태은 (2008). 학교행복 검사도구 개발 및 타당화. 교육심리연구, 22(1), 259- 279.
김혜진, 김옥분 (2009). Rasch 측정모형의 적용을 통한 대학생용 단축형 완벽주의 척도 구성. 교육평가연구, 22(3), 757-781.
문은식 (2013). 중학생이 지각한 교실의 사회적 환경, 동기적 신념, 관여 및 학업성취도의 구조적 관계. 아동교육, 22(4), 25-43.
박혜숙 (2013). Rasch 측정모형을 사용한 대학생 대상 다문화 수용성 척도개발 및 타당화. 교육심리연구, 27(2), 453-477.
설현수 (2007). Messick의 타당도 관점에서 Rasch 측정모형 적용을 통한 대학 강의평가도구 개발의 타당화. 교육평가연구, 20(4), 31-51.
설현수, 김성은, 김동민 (2005). Rasch 모형을 이용한 Marlowe-Crowne의 사회적 정향성 척도 타당화. 교육평가연구, 18(1), 101- 123.
손원숙 (2003). 심리검사 번안에 대한 통합적 접근. 한국심리학회지: 일반, 22(2), 57-80.
신현숙 (2014). 중학생이 지각한 교실맥락과 인지적 참여의 관계에서 행동적 및 정서적 참여의 중다매개효과. 청소년학연구, 21(4), 83-106.
신현숙 (2015. 8.). 초․중․고등학생의 학습활동참여에 대한 잠재평균분석. 2015 한국심리학회 연차학술대회 자료집 (p. 429). 서울.
신현숙 (2017). 중학생이 지각한 학급맥락과 학업유능성 및 주관적 안녕감의 관계: 기본심리욕구 충족과 수업참여의 매개효과. 아동교육, 26(2), 41-65.
신현숙, 염시창 (2013). 학교참여 다차원 구인에 대한 성별 및 학교급별 잠재평균분석. 한국심리학회지, 학교, 10(2), 283-305.
신현숙, 염시창 (2015). 중학생용 한국형 학교생활참여척도의 개발 및 타당화. 한국심리학회지: 학교, 12(2), 177-202.
이승연, 한미리 (2014). 정신건강 2요인 모델에 따른 중학생의 학교참여와 학업성취. 청소년학연구, 21(10), 195-218.
이자영, 이상민 (2012). 한국형학업열의척도 개발 및 타당화. 교육방법연구, 24(1), 131- 147.
추헌택, 손원숙 (2012). 대학생용 학업참여 척도(UWES-S)의 타당화: 학업동기, 참여 및 만족도의 구조적 관계. 한국심리학회지: 학교, 9(3), 485-503.
홍세희, 조용래 (2006). 역기능적 신념검사 단축판의 구성: Rasch 평정척도모형의 적용. 한국심리학회지: 임상, 25(3), 865-880.
Alexander, K. L., Entwisle, D. R., & Horsey, C. S. (1997). From first grade forward: Early foundations of high school dropout. Sociology of Education, 70, 87-107.
Anderman, E. M., Maehr, M. L., & Midgley, C. (1999). Declining motivation after the transition to middle school: Schools can make a difference. Journal of Research and Development in Education, 32, 131-147.
Andrich, D. (1978). Rating formulation for ordered response categories. Psychometrika, 43(4), 561- 573.
Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools, 45(5), 369-386.
Appleton, J. J., Christenson, S. L., Kim D., & Reschly, A, L, (2006). Measuring cognitive and psychological engagement: Validation of the student engagement instrument. Journal of School Psychology, 44, 427-445.
Betts, J. E., Appleton, J. J., Reschly, A. L., Christenson, S. L., & Huebner, E. S. (2010). A study of the factorial invariance of the student engagement instrument (SEI): Results from middle and high school students. School Psychology Quarterly, 25(2), 84-93.
Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd. ed.). New York: Routledge.
Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.). Testing structural equation models (pp. 136-162). Newbury Park, CA: Sage.
Cole, J. C., Rabin, A. S., Smith, T. L., & Kaufman, A. S. (2004). Development and validation of a Rasch-derived CES-D short form. Psychological Assessment, 16(4), 360-372.
Cornell, D. G., Lovegrove, P. J., & Baly, M. W. (2014). Invalid survey response patterns among middle school students. Psychological Assessment, 26(1), 277-287.
Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Erlbaum.
Fall, A., & Roberts, G. (2012). High school dropouts: Interactions between social context, self-perceptions, school engagement, and student dropout. Journal of Adolescence, 35, 787-798.
Finn, J. D. (1989). Withdrawing from school. Review of Educational Research, 59, 117-142.
Fox, C. M., & Jones, J. A. (1998). Use of Rasch modeling in counseling psychology research. Journal of Counseling Psychology, 45(1), 30-45.
Fredricks, J. A. (2011). Engagement in school and out-of-school contexts: A multidimensional view of engagement. Theory Into Practice, 50, 327-335.
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59-109.
Fredrickson, B. L. (2001). The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American Psychologist, 58, 218-226,
Glanville, J. L., & Wildhagen, T. (2007). The measurement of school engagement: Assessing dimensionality and measurement invariance across race and ethnicity. Educational and Psychological Measurement, 67(6), 1019-1041.
Griffiths, A., Sharkey, J. D., & Furlong, M. J. (2009). Student engagement and positive school adaptation. In R. Gilman, E. S. Huebner, & M. J. Furlong (Eds.), Handbook of positive psychology in schools (pp. 197-211). New York: Routledge.
Hong, S., Kim, B., & Wolfe, M. (2005). A psychological revision of the European American Values Scale for Asian Americans using the Rasch model. Measurement and Evaluation in Counseling and Development, 37(4), 194-207.
Hu, L., & Bentler, P. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55.
Jimerson, S. R., Campos, E., & Greif, J. L. (2003). Toward an understanding of definitions and measures of school engagement and related terms. The California School Psychologist, 8, 7-27.
Lam, S., Jimerson, S., Shin, H., Cefai, C., Veiga, F. H., Hatzichristou, C., et al. (2016). Cultural universality and specificity of student engagement in school: The results of an international study from 12 countries. British Journal of Educational Psychology, 86, 137-153.
Li, Y., Zhang, W., Liu, J., Arbeit, M. R., Schwartz, S. J., Bowers, E. P., & Lerner, R. M. (2011). The role of school engagement in preventing adolescent delinquency and substance use: A survival analysis. Journal of Adolescence, 34, 1181-1192.
Linacre, J. M. (2004). Optimizing rating scale category effectiveness. In E. V. Smith, Jr., & R. M. Smith (Eds.), Introduction to Rasch measurement: Theory, models and application (pp. 258-278). Maple Grove, MN: JAM Press.
Linacre, J. M. (2017a). A user’s guide to WINSTEPS: Rasch-model computer program. www.winsteps.com.
Linacre, J. M. (2017b). WINSTEPS: Computer Program 3.93. www.winsteps.com.
Marks, H. M. (2000). Student engagement in instructional activity: Patterns in the elementary, middle, and high school years. American Educational Research Journal, 37, 153-184.
Martinez, R. S., Aricak, O. T., Graves, M. N., Peters-Myszak, J., & Nellis, L. (2011). Changes in perceived social support and socioemotional adjustment across the elementary to junior high school transition. Journal of Youth & Adolescence, 40, 519-530.
Muthén, L. K, & Muthén, B. O. (2012a). Mplus user’s guide (7th ed.). Los Angeles: Muthén & Muthén.
Muthén, L. K, & Muthén, B. O. (2012b). Mplus: Computer Program 7.0. www.statmodel.com.
Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33-40.
Reeve, J. (2013). How students create motivationally supportive learning environments for themselves: The concept of agentic engagement. Journal of Educational Psychology, 105, 575-595.
Reeve, J., & Tseng, C. (2011). Agency as a fourth aspect of students’ engagement during learning activities. Contemporary Educational Psychology, 36, 257-267.
Salmela-Aro, K., & Upadyada, K. (2012). The schoolwork engagement inventory: Energy, dedication, and absorption (EDA). European Journal of Psychological Assessment, 28(1), 60-67.
Schaufeli, W. B., Martinez, I. M., Pinto, A. M., Salanova, M., & Baller, A. B. (2002a). Burnout and engagement in university students: A cross-national study. Journal of Cross-Cultural Psychology, 33(5), 464-481.
Schaufeli, W. B., Salanova, M., Gonzalez-Roma, V., & Bakker, A. B. (2002b). The measurement of engagement and burnout: A two sample confirmatory factor analytic approach. Journal of Happiness Studies, 3, 71-92.
Shernoff, D. J., Csikszentmihalyi, M., Schneider, B., & Shernoff, E. S. (2003). Student engagement in high school classrooms from the perspective of flow theory. School Psychology Quarterly, 18(2), 158-176.
Simons-Morton, B., & Chen, R. (2009). Peer and parent influences on school engagement among early adolescents. Youth & Society, 41(1), 3-25.
Skinner, E., Furrer, C., Marchand, G., & Kindermann, T. (2008). Engagement and disaffection in the classroom: Part of a larger motivational dynamic? Journal of Educational Psychology, 100(4), 765-781.
Upadyada, K., & Salmela-Aro, K. (2013). Development of school engagement in association with academic success and well-being in varying social contexts. European Psychologist, 18(2), 136-147.
Wang, M., Willett, J. B., & Eccles, J. S. (2011). The assessment of school engagement: Examining dimensionality and measurement invariance by gender and race/ethnicity. Journal of School Psychology, 49, 465-480.
Wolfe, E. W., & Smith, E. V., Jr. (2007). Instrument development tools and activities for measure validation using Rasch models: Part Ⅱ. Validation activity. In E. V. Smith, Jr., & R. M. Smith (Eds.), Rasch measurement: Advanced and specialized applications (pp. 202-242). Maple Grove, MN: JAM Press.
Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8, 370-371.
Yazzie-Mintz, E., & McCormick, K. (2012). Finding the humanity in the data: Understanding, measuring, and strengthening student engagement. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 743-760). New York: Springer.
You, S., & Sharkey, J. (2009). Testing a developmental-ecological model of student engagement: A multilevel latent growth curve analysis. Educational Psychology, 29(6), 659-684.
Zhu, W., Timm, G., & Ainsworth, B. (2001). Rasch calibration and optimal categorization of an instrument measuring women’s exercise perseverance and barriers. Research Quarterly for Excercise and Sport, 72(2), 104-116.
- Downloaded
- Viewed
- 0KCI Citations
- 0WOS Citations