Abstract

The present study starts with hypotheses that response instructions and scoring alternatives influence construct validities, criterion-related validities and incremental validities. The results show possibilities that validities could vary with response instructions and scoring scoring alternatives. In a situational judgment test there are two types of instructions: One is to draw knowledge responses and the other is to draw behavioral tendency responses. In scoring the tests a scenario can be the unit or a response alternative under a scenario can be the unit. First, construct validities can be different depending on response instructions. Second, knowledge response instructions are more correlated with task performance when a scenario is the scoring unit and behavioral tendency response instructions with contextual performance when response alternatives are scored. Third, meaningful incremental validities are observed for each of response instructions. Finally the reasons why effectiveness of response instructions were not clearly differentiated are discussed.