바로가기메뉴

본문 바로가기 주메뉴 바로가기

logo

ISSN : 1226-9654

Article Contents

Prev Next
e-Submission

Vol.30 No.3

Citation Share

Cross-modal Correspondence Between Acoustic Feature and Shape

The Korean Journal of Cognitive and Biological Psychology / The Korean Journal of Cognitive and Biological Psychology, (P)1226-9654; (E)2733-466X
2018, v.30 no.3, pp.269-275
https://doi.org/10.22172/cogbio.2018.30.3.005


& (2018). Cross-modal Correspondence Between Acoustic Feature and Shape. The Korean Journal of Cognitive and Biological Psychology, 30(3), 269-275, https://doi.org/10.22172/cogbio.2018.30.3.005

Abstract

Our brain tends to associate stimulus features across the senses in a non-random manner. For example, people show consistency in labelling a rounded shape ‘maluma’/‘bouba’ and a spiky shape ‘takete’/‘kiki’. Previous studies have attributed this phenomenon to the correspondence between sound and shape, but without controlling for other potential factors (i.e., linguistic/orthographical factors). The present study examines the role of acoustic aspect per se by manipulating articulatory gestures to generate synthetic speech sounds not confined to a specific language. Participants were asked to choose either a rounded or spiky shape to indicate the shape that better matched each synthetic speech sound. The results demonstrate that shape choice was systematically mapped on to the dimensions manipulated to generate the sounds. These results indicate that acoustic features indeed drive the association between sound and visual shape.

keywords
cross-modal correspondence, sound, articulatory synthesis, shape, 교차양태 관련성, 소리, 조음 합성, 형태

Reference

1.

Ahlner, F., & Zlatev, J. (2010). Cross-modal iconicity: A cognitive semiotic approach to sound symbolism. Sign Systems Studies, 38(1/4), 298-348.

2.

Ashby, F. G., Maddox, W. T., & Lee, W. W. (1994). On the dangers of averaging across subjects when using multidimensional scaling or the similarity-choice model. Psychological Science, 5, 144-151.

3.

Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10, 433-436.

4.

Bremner, A. J., Caparos, S., Davidoff, J., de Fockert, J., Linnell, K. J., & Spence, C. (2013). “Bouba” and “Kiki” in Namibia? A remote culture make similar shape-sound matches, but different shape-taste matches to Westerners. Cognition, 126, 165-172.

5.

Cox, T. F., & Cox, M. A. (2001). Multidimensional scaling (2nd ed.). London, UK: Chapman & Hall.

6.

Cuskley, C., Simner, J., & Kirby, S. (2017). Phonological and orthographic influences in the bouba-kiki effect. Psychological Research, 81, 119-130.

7.

Davis, R. (1961). The fitness of names to drawings. A cross-cultural study in Tanganyika. British Journal of Psychology, 52, 259-268.

8.

Donaldson, G. S., & Kreft, H. A. (2006). Effects of vowel context on the recognition of initial and medial consonants by cochlear implant users. Ear and Hearing, 27, 658-677.

9.

D’Onofrio, A. (2014). Phonetic detail and dimensionality in sound-shape correspondences: Refining the Bouba-Kiki paradigm. Language and Speech, 57, 367-393.

10.

Fort, M., Martin, A., & Peperkamp, S. (2015). Consonants are more important than vowels in the Bouba-kiki Effect. Language and Speech, 58, 247-266.

11.

Fort, M., Weiß, A., Martin, A., Peperkamp, S. (2013). Looking for the bouba-kiki effect in pre-lexical infants. in: Proceedings of the 12th international conference on auditory-visual speech processing, 71-76.

12.

Gaißert, N., Wallraven, C., & Bülthoff, H. H. (2010). Visual and haptic perceptual spaces show high similarity in humans. Journal of Vision, 10, 1-20.

13.

Hinton, L., Nichols, J., & Ohala, J. J. (1994). Sound Symbolism. Cambridge, UK: Cambridge University Press.

14.

Köhler, W. (1929). Gestalt psychology. New York: Liveright.

15.

Köhler, W. (1947). Gestalt psychology (2nded.). NewYork: Liveright.

16.

Lee Masson, H., Bulthé, J., Op de Beeck, H. P., & Wallraven, C. (2016). Visual and haptic shape processing in the human brain: Unisensory processing, multisensory convergence, and top-down influences. Cerebral Cortex, 26, 3402-3412.

17.

Maurer, D., Pathman, T., & Mondloch, C. J. (2006). The shape of boubas: Sound-shape correspondences in toddlers and adults. Developmental Science, 9, 316-322.

18.

Nam, H., Goldstein, L. M., Giulivi, S., Levitt, A. G., Whalen, D. H. (2013). Computational simulation of CV combination preferences in babbling. Journal of Phonetics, 41, 63-77.

19.

Nam, H., Goldstein, L. M., Saltzman, E., & Byrd, D. (2004). TADA: An enhanced, portable Task Dynamics model in MATLAB. The Journal of the Acoustical Society of America, 115, 2430-2430.

20.

Nielsen, A., & Rendall, D. (2011). The sound of round: Evaluating the sound-symbolic role of consonants in the classic Takete-Maluma phenomenon. Canadian Journal of Experimental Psychology, 65, 115-124.

21.

Nielsen, A., & Rendall, D. (2013). Parsing the role of consonants versus vowels in the classic Takete-Maluma phenomenon. Canadian Journal of Experimental Psychology, 67, 153-163.

22.

Ozturk, O., Krehm, M., & Vouloumanos, A. (2013). Sound symbolism in infancy: Evidence for sound-shape cross-modal correspondences in 4-month-olds. Journal of Experimental Child Psychology, 114, 173-186.

23.

Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437-442.

24.

Ramachandran, V. S., & Hubbard, E. M. (2001). Synaesthesia - a window into perception, thought and language. Journal of Consciousness Studies, 8, 3-34.

25.

Spence, C. (2011). Crossmodal correspondences: A tutorial review. Attention, Perception, & Psychophysics, 73, 971-995.

26.

Styles, S. J., & Gawne, L. (2017). When does Maluma/Takete fail? Two key failures and a meta-analysis suggest that phonology and phonotactics matter. i-Perception, 8, 1-17.

27.

Taitz, A., Assaneo, M. F., Elisei, N., Trípodi, M., Cohen, L., Sitt, J. D., & Trevisan, M. A. (2018). The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation. PLoS ONE, 13, e0193466.

28.

Tarte, R. (1974). Phonetic symbolism in adult native speakers of Czech. Language and Speech, 17, 87-94.

The Korean Journal of Cognitive and Biological Psychology

e-Submission OA Policy