ISSN : 1229-067X
Many of the psychological processes are revealed in language responses. Verbal reports in the form of Think-Aloud protocols, self-explanations, or responses to various questions have been successfully used to understand psychological processes that are difficult to capture by utilizing traditional measures such as reaction time and Likert-scale responses. The goal of this paper is to introduce a particular method to analyze verbal data for scientific research of psychology. Called as verbal data analysis, this method is designed to systematically analyze verbal data to produce objective and quantitative results. The paper describes core features of the method along with research examples in which verbal data analysis was successfully applied. Detailed procedures of the analysis are then described with issues that researchers need to be aware of at each stage of data collection and analysis. Lastly, verbal data analysis is compared to other similar methods. Strengths and weaknesses of the method are also discussed.
이정모. (2001). 인지심리학: 형성사, 개념적 기초, 조망. 아카넷. 서울.
조명한, 이정모, 김정오, 신현정, 이광오, 도경수, 이양, 이현진, 김영진, 김소영, 고성룡, & 정혜선. (2003). 언어 심리학. 학지사. 서울.
조용환. (1999). 질적연구: 방법과 사례. 교육과학사. 서울.
Azevedo, R., & Cromley, J. G. (2004). Does training on self-regulated learning facilitate students' learning with hypermedia? Journal of Educational Psychology, 96(3), 523-535.
Baker, M., Andriessen, J., Lund, K., van Amelsvoort, M., & Quignard, M. (2007). Rainbow: A framework for analyzing computer-mediated pedagogical debates. International Journal of Computer-Supported Collaborative Learning, 2, 315-357.
Benbunan-Fich, R. (2001). Using protocol analysis to evaluate the usability of a commercial web site. Information & Management, 39, 151-163.
Berthold, K., Nückles, M., & Renkl, A. (2007). Do learning protocols support learning strategies and outcomes? The role of cognitive and metacognitive prompts. Learning and Instruction, 17(5), 564-577.
Bosshardt, H.-G. (2006). Cognitive processing load as a determinant of stuttering: summary of a research programme. Clinical Linguistics & Phonetics, 20(5), 371-85.
Bryman, A. (1984). The debate about quantitative and qualitative research: A question of method or epistemology. The British Journal of Sociology, 35(1), 75-92.
Chan, J., Fu, K., Schunn, C., Cagan, J., Wood, K., & Kotovsky, K. (2011). On the benefits and pitfalls of analogies for innovative design: Ideation performance based on analogical distance, commonness, and modality of examples. Journal of Mechanical Design, 133(8), 081004.
Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive Psychology, 4, 55-81.
Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The Journal of the Learning Science, 6(3), 271-315.
Chi, M. T. H. (2006). Laboratory methods for assessing experts' and novices' knowledge. In K. A. Ericsson, N. Charness, P. Feltovich & R. Hoffman (Eds.), Cambridge Handbook of Expertise and Expert Performance (pp.167-184). Cambridge: Cambridge University Press.
Chi, M. T. H., & Koeske, R. D. (1983). Network representation of a child's dinosaur knowledge. Developmental Psychology, 19(1), 29-39.
Chi, M. T. H., Bassok, M., Lewis, M. T., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182.
Chi, M. T. H., de Leeuw, N., Chiu, M., & LaVancher, C. (1994). Eliciting self -explanations improves understanding. Cognitive Science, 18, 439-477.
Clark, H. H., & Fox Tree, J. E. (2002). Using uh and um in spontaneous speaking, Cognition, 84, 73-111.
Coleman, E. B., Brown, A. L., & Rivkin, I. D. (1997). The effect of instructional explanations on learning from scientific texts. The Journal of the Learning Sciences, 6(4), 347-365.
De Wever, B., Schellens, T., Valcke, M., & van Keer, H. (2006). Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Computers and Education, 46, 6-28.
Dunbar, K. (1997). How Scientists Think: On-line creativity and conceptual change in science. In T. B. Ward, S. M. Smith & J. Vaid (Eds.), Creative thought: An investigation of conceptual structures and processes. (pp.461-493). Washington: American Psychological Association.
Dyke, G., Lund, K., & Girardot, J.-J. (2009). Tatiana: An environment to support the CSCL analysis process. Paper presented at the International Conference of the Learning Sciences, Rhodes, Greece.
Ericsson, K. A. (2000). Expertise in interpreting: An expert-performance perspective. Interpreting, 5(2), 187-220.
Ericsson, K. A. (2006). Protocol analysis and expert thought: Concurrent verbalizations of thinking during experts' performance on representative tasks. In K. A. Ericsson, N. Charness, P. Feltovich & R. Hoffman (Eds.), Cambridge Handbook of Expertise and Expert Performance. Cambridge: Cambridge University Press.
Ericsson, K. A., Delaney, P. F., & Weaver, G. (2004). Uncovering the structure of a memorist’s superior “basic” memory capacity, Cognitive Psychology, 49, 191-237.
Ericsson, K. A., & Simon, H. A. (1980). Verbal report as data. Psychological Review, 87(3), 215-251.
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal report as data. Cambridge, MA: The MIT Press.
Eye, A. V., & Mun, E. Y. (2005). Analyzing rater agreement. Mahwah, NJ: Earlbaum.
Flower, L. S., & Hayes, J. R. (1981). A cognitive process theory of writing. College Composition and Communication, 32(4), 365-387.
Fox, M. C., Ericsson, K. A., & Best, R. (2011). Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods. Psychological Bulletin, 137(2), 316-44.
Gee, J. P., & Green, J. L. (1998). Discourse analysis, learning, and school practice: A methodological study. In Review of Research in Education (Vol. 23). Washington, DC: AERA.
Jordan, B., & Henderson, A. (1995). Interaction analysis: Foundations and practice. The Journal of The Learning Science, 4(1), 39-103.
Hoffman, R. R., Shadbolt, N. R., Burton, A. M., & Klein, G. (1995). Eliciting knowledge from experts: A methodological analysis. Organizational Behavior and Human Decision Processes, 62, 129-158.
Holsti, O. R. (1969). Content analysis for the social sciences and humanities. Menlo Park, CA: Addison-Wesley Publishing Company.
Hsiu-Fang, H., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277-1288.
Jefferson, G. (2004). Glossary of transcript symbols with an introduction. In G. H. Lerner (Ed.), Conversation analysis: Studies from the first generation (pp.13-31). Amsterdam: John Benjamin Publishing Company.
Jeong, H. (2013). Verbal data analysis for understanding interactions. In C. E. Hmelo-Silver, A. M. O’Donnell, C. Chan, & C. A. Chinn (Eds.), The international handbook of collaborative learning. London: Taylor & Francis.
Jeong, H., Chen, W., & Looi, C. K. (2011). Analysis of group understanding in artifact-mediated discourse. In the Proceedings of Computer-Support for Collaborative Learning.
Jeong, H., & Chi, M. H. (2007). Knowledge convergence and collaborative learning. Instructional Science, 35, 287-315.
Johnson, R., & Christensen, L. (2008). Educational research: Quantitative, qualitative, and mixed approaches. Thousand Oaks, CA: Sage.
Kaplan, C. K. , & Simon, H. A. (1990). In search of insight. Cognitive Psychology, 22, 374-419.
Kimmerle, J., Moskaliuk, J., & Cress, U. (2011). Using Wikis for Learning and Knowledge Building: Results of an Experimental Study. Educational Technology & Society, 14, 138-148.
Krampe, R. T., & Charness, N. (2006). Aging and expertise. In K. A. Ericsson, N. Charness, P. Feltovich, & R. Hoffman (Eds.), Cambridge Handbook of Expertise and Expert Performance (pp.723-742). Cambridge, UK: Camdridge University Press.
Krippendorff, K. (2004). Content analysis: An introduction to its methodology. Thousand Oaks, CA: Sage Publications.
McNamara, D. S. (2004). SERT: Self-explanation reading training. Discourse Processes, 38(1), 1-30.
Meier, A., Spada, H., & Rummel, N. (2007). A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning, 2, 63-86.
Neuendorf, K. A. (2002). The content analysis: Guidebook. London: Thousand Oaks.
Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.
Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231-259.
Opfer, J. E., & Siegler, R. S. (2004). Revisiting preschoolers' living things concept: A microgenetic analysis of conceptual change in basic biology. Cognitive Psychology, 49, 301-332.
Okada, T., & Simon, H. A. (1997). Collaborative discovery in a scientific domain. Cognitive Sciences, 21(2), 109-146.
Paletz, S. B. F., Schunn, C. D., & Kim, K. H. (2012). The interplay of conflict and analogy in multidisciplinary teams. Cognition, 126, 1- 19.
Payne, J. W. (1976). Task complexity and contingent processing in decision making: An information search and protocol analysis. Organizational Behavior and Human Performance, 16(2), 366-387.
Pennebaker, J. W., Booth, R. J., & Francis, M. E. (2007). Operator's manual: Linguistic inquiry and word count: LIWC2007. Austin, TX: LIWC.net.
Pennebaker, J. W., Mehl, M. R., & Niederhoffer, K. G. (2003). Psychological aspects of natural language use: our words, our selves. Annual Review of Psychology, 54, 547-77.
Pressley, M., & Afflerbach, P. (1995). Verbal protocols of reading: The nature of constructively responsive reading. Hillsdale, NJ: Erlbaum.
Rattleff, P. (2007). The reliability of content analysis of computer conference communication. Computers and Education, 49, 230-242.
Renkl, A. (1997). Learning from worked-out examples: A study on individual difference. Cognitive Science, 21(1), 1-29.
Renkl, A. (2002). Worked-out examples: Instructional explanations support learning by self-explanations. Learning and Instruction, 12, 529-556.
Renkl, A., Stark, R., Gruber, H., & Mandl, H. (1998). Learning from worked-out examples: The effects of example variability and elicited self-explanations. Contemporary Educational Psychology, 23(23), 90-108.
Rittle-johnson, B. (2006). Promoting transfer: Effects of self-explanation and direct instruction, Child Development, 77(1), 1-15.
Roscoe, R. D. (2013). Self-monitoring and knowledge-building in learning by teaching. Instructional Science, 42(3), 327-351.
Rosé, C. (2008). Analyzing collaborative learning processes automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning. International Journal of Computer-Supported Collaborative Learning, 3, 237-271.
Sacks, H. (1992). Lectures on conversation. Malden, MA: Blackwell.
Schneider, J., Passant, A., & Breslin, J. G. (2011). Understanding and Improving Wikipedia Article Discussion Spaces. In SAC’11.
Schooler, J. W., & Engstler-Schooler, T. Y. (1990). Verbal overshadowing of visual memories: Some things are better left unsaid. Cognitive Psychology, 22(1), 36-71.
Schooler, J. S., Ohlsson, S., & Brooks, K. (1993). Thoughts beyond words: When language overshadows insight. Journal of Experimental Psychology: General, 122(2), 166-183.
Siegler, R. S. (1995). How does change occur: A microgenetic study of number conservation. Cognitive Psychology, 28, 225-273.
Simon, D. P., & Simon, H. A. (1978). Individual differences in problem solving physics problem. In R. S. Siegler (Ed.), Children's thinking: What develops? Hillsdale, NJ: Erlbaum.
Stempflea, J., & Badke-Schaub, P. (2002). Thinking in design teams - an analysis of team communication. Design Studies, 23(5), 473-496.
Straus, A., & Corbin, J. (1990). Basics of qualitative research. Newbury Park: Sage publication.
Strijbos, J., Martens, R. L., Prins, F. J., & Jochems, W. M. G. (2006). Content analysis: What are they talking about? Computers and Education, 46, 29-48.
Suwa, M., & Tversky, B. (1997). What do architects and students perceive in their design sketches? A protocol analysis. Design Studies, 18(4), 385-403.
van Aalst, J. (2009). Distinguishing knowledge- sharing, knowledge-construction, and knowledge -creation discourse. International Journal of Computer-Supported Collaborative Learning, 4, 259-287.
Veenman, M. V. J., Prins, F. J., & Verheij, J. (2003). Learning styles: self-reports versus thinking-aloud measures. The British Journal of Educational Psychology, 73(Pt 3), 357-72.
Vosniadou, S., & Brewer, W. F. (1992). Mental models of the earth: A study of conceptual change in childhood. Cognitive Psychology, 24, 535-585.
Wineburg, S. S. (1991). Historical problem solving: A study of the cognitive processes used in the evaluation of documentary and pictorial evidence. Journal of Educational Psychology, 83(1), 73-87.
Wineburg, S. (1998). Reading Abraham Lincoln: An expert/expert study in the interpretation of historical texts. Cognitive Science, 22(3), 319- 346.