ISSN : 1229-0696
The purpose of this study is to examine the different criterion-related validities from six different combinations of three keying methods(SME consensus, average in response and empirical keying) and two scoring methods(-2~2, B-W) using a leadership situational judgment test with knowledge instructions. The test was administered to 395 employees who has managerial positions in a Korean company after developing items and deciding on keys. The results reveal that the combination of SME × B-W has the highest criterion-related validity among the six combinations. This study suggests that the keying method of a situational judgement test is one of the important factors that determine the psychometric and predictive aspects of the test.
강민우, 윤창영, 이순묵 (2005). 지시문과 채점 방식에 따른 상황판단검사의 타당도 비교. 한국심리학회지: 산업 및 조직, 18(3), 547- 565.
이상철, 이순묵, 조영일 (2003). 지필형 상황판단검사에 대한 비평적 고찰. 한국심리학회지: 산업 및 조직, 16(3), 129-154.
박동건, 전인식 (2001). 전기자료(Biodata) 문항의 가중치 부여 체계간의 타당도 연구: 분석집단 크기에 따른 비교연구. 한국심리학회지: 산업 및 조직, 14(1), 101-113.
Borman, W. C., & Motowidlo, S. J. (1993). Expanding the criterion domain to include elements of contextual performance. In N. Schmitt & W. Borman(Eds.), Personnel selection in organizations(pp.71-98). New York: Jossey- Bass.
Borman, W. C., White, L. A., Pulakos, E. D., & Oppler, S. H. (1991). Models of supervisory job performance ratings. Journal of Applied Psychology, 76, 863-872.
Bruce, M. M., & Learner, D. B. (1958). A supervisory practices test. Personnel Psychology, 11, 207-216.
Cardall, A. J. (1942). Preliminary manual for the Test of Practical Judgment. Chicago: Science Research.
Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82, 143-159.
Chan, D., & Schmitt, N. (2002). Situational Judgment and Job Performance. Human Performance, 15(3), 233-254
Clevenger, J., Pereira, G. M., Wiechmann, D., Schmitt, N., & Harvey, V. S. (2001). Incremental validity of situational judgment tests. Journal of Applied Psychology, 86(3), 410-417.
Conn, S. R., & Rieke, M. L. (1994). The 16PF fifth edition technical manual. Champaign, IL: Institute for personality and ability Testing
File, Q. W. (1945). The measurement of supervisory quality in industry. Journal of Applied Psychology, 29, 381-387.
File, Q. W., & Remmers, H. H. (1948). How Supervise? manual 1948 revision. New York: Psychological Corporation.
Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51, 327-358
In J. P. Campbell & D. J. Knapp(Eds.), Exploring the limits in personnel selection and classfication. Mahwah, N. J.: Lawrence Erlbaum Associates.
Greenberg, S. H. (1963). Supervisory Judgment Test manual. Washington, DC: U. S. Civil Service Commission.
Knapp, D. J., Campbell, C. H., Borman, W. C., Pulakos, E. D., & Hanson, M. A. (2001). Performance assessment for a population of jobs. In J. P. Campbell & D. J. Knapp (Eds.), Exploring the limits in personnel selection and classfication. Mahwah, N. J.: Lawrence Erlbaum Associates.
Latham, G. P. (1989). The reliability, validity, and practicality of the situational interview. In R. W. Eder G. R. Ferris(Eds.), The employment interview: theory, research, and practice(pp. 169-182). Newbury Park, CA: Sage.
McDaniel, M. A., Hartman, N. S., & Grubb Ⅲ, W. L. (2003, April). Situational Judgment Tests, Knowledge, Behavioral Tendency, and Validity: A Meta-Analysis. Paper presented at the 18th Annual Conference of the Society for Industrial and Oraganizational Psychology, Orlando, FL.
McDaniel, M. A., Hartman, N. S., Whetzel, D. L., & Grubb Ⅲ, W. L. (2007). Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel psychology, 60, 63-91.
McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: a clarification of the literature. Journal of Applied Psychology. 86(4), 730-740.
McDaniel, M. A., & Nguyen, N, T. (2001). Situational judgment tests: a review of practice and constructs assessed. International Journal of Selection and Assessment. 9, 103-113.
McDaniel, M. A., Yost, A. P., Ludwick, M. H., Hense, R. L., & Hartman, N. S. (2004, April). Incremental validity of a situational judgement test. Paper presented at the 19th Annual Conference of the Society for Industrial and Oraganizational Psychology, Chicago.
Motowidlo, S. J. (2000). Some basic issues related to contextual performance and organizational citizenship behavior in human resource management. Human Resource Management Review, 10, 115-126.
Motowidlo, S. J., & Tippins, N. (1993). Further studies of the low-fidelity simulation in the form of a situation inventory. Journal of Occupational and Organizational Psychology, 66, 337-344.
Motowidlo, S. J., & Van Scotter, J. R. (1994). Evidence that task performance should be distinguished from contextual performance. Journal of Applied Psychology. 79(4), 475-480.
Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative seleciton procedure: the low-fidelity situation. Journal of Applied Psychology, 75(6), 640-647.
Motowidlo, S. J., Hanson, M. A., & Crafts, J. L. (1997). Low-fidelity simulations. In D. L. Whetzel & G. R. Wheaton(Eds.), Applied measurement methods in industrial psychology. Palo Alto, CA: Davies-Black Publishing.
Motowidlo, S.. J., Borman, W., & Schmit, M. (1997). A theory of individual differences in task and contextual performance. Human Performance, 10, 71-83.
Oswald, F. L., Schmitt, N., Kim. B. H., Ramsay, L. J., & Gillespie, M. A. (2004). Developing a biodata measure and situational judgement inventory as predictors of college student performance. Journal of Applied Psychology, 89(2), 187-207.
Parker, C.W., Golden III, J. H., Russell, D.P. & Redmond, M. R. (2000). The development of a construct-related scoring key of a situational judgment inventory for enhancing criterion-related validity. Paper presented at the 15th annual conference of the Society of Industrial and Organizational Psychology, New Orleans, April.
Ployhart, R. E., & Ehrhart, M. G., (2003). Be careful what you ask for: effects of response instructions on the construct validity and reliability of situational judgment tests. International Journal of Selection and Assessment. 11(1), 1-16.
Pulakos, E. D., & Schmitt, N. (1996). An evaluation of two strategies for reducing adverse impact and their effects on criterion-related validity. Human Performance, 9, 241-258.
Reynolds, D. H., Sydell, E. J., Scott, D. R., & Winter, J. L. (2000, April). Factors affecting situational judgment test characteristics. Paper presented at the 15th Annual Conference of the Society for Industrial and Organizational Psychology. New Orleans, LA.
Smith, C. A., Organ, D. W., & Near, J. P. (1993). Organizational citizenship behavior: Its nature and antecedents. Journal of Applied Psychology, 68, 653-663.
Smith, K. C., & McDaniel, M. A. (1998). Criterion and construct validity evidence for a situational judgment measure. Paper presented at the 13th Annual Conference of the Society for Industrial and Organizational Psychology, Dallas, TX.
Wagner, R. K. (1987). Tacit knowledge in everyday intelligent behavior. Journal of Personality and Social Psychology, 52, 1236-1247
Wagner, R. K., & Sternberg, R. J. (1985). Practical intelligence in real-world pursuits: the role of tacit knowledge. Journal of Personality and Social Psychology, 49(2), 436-458
Waugh, G. (2002). Selecting response options and items for a situational judgment test. Paper presented as part of the following symposium - Understanding and Predicting Performance in Future Jobs. 17th Annual Conference of the Society for Industrial and Organizational Psychology, Toronto.
Weekley, J. A., & Jones, C. (1997). Video-based situational testing. Personnel Psychology, 50, 25-49.
Weekley, J. A., & Jones, C. (1999). Further studies of situational tests. Personnel Psychology, 52, 679-700.