바로가기메뉴

본문 바로가기 주메뉴 바로가기

logo

Nasal Sounds are Lighter and More Yellowish than Glottal Sounds: Cross-modal Associations between Consonant Sounds and Colors

The Korean Journal of Cognitive and Biological Psychology / The Korean Journal of Cognitive and Biological Psychology, (P)1226-9654; (E)2733-466X
2020, v.32 no.1, pp.85-99
https://doi.org/10.22172/cogbio.2020.32.1.006



Abstract

People share implicit cross-modal mappings for certain visual and auditory features such as pitch and human speech. The current study explored the role of phonetic features in intrinsic associations between consonant sounds and colors. For this purpose, we presented synthetic consonant sounds generated by parametrically manipulating oral and non-oral constriction gestures of speech organs, using an articulatory synthesizer. Participants were asked to choose a color after hearing each sound. Color-matching results showed that nasal sounds characterized by a velic gesture were associated with lighter and more yellowish colors than other sounds. The perceptual space of the consonant sounds from dissimilarity judgment ratings indicated that participants could capture the consonantal nature of the stimuli. These results imply the non-arbitrary association between phonetic features of consonants and colors.

keywords
cross-modal association, color, consonant, articulatory synthesis, 교차감각 연합, 색, 자음, 조음 합성

Reference

1.

Abramson, A. S., Nye, P. W., Henderson, J. B., & Marshall, C. W. (1981). Vowel height and the perception of consonantal nasality. The Journal of the Acoustical Society of America, 70, 329-339.

2.

Asano, M., Imai, M., Kita, S., Kitajo, K., Okada, H., & Thierry, G. (2015). Sound symbolism scaffolds language development in preverbal infants. Cortex, 63, 196-205.

3.

Auracher, J., Albers, S., Zhai, Y., Gareeva, G., & Stavniychuk, T. (2010). P is for happiness, N is for sadness: Universals in sound iconicity to detect emotions in poetry. Discourse Processes, 48, 1-25.

4.

Ashby, F. G., Maddox, W. T., & Lee, W. W. (1994). On the dangers of averaging across subjects when using multidimensional scaling or the similarity-choice model. Psychological Science, 5, 144-151.

5.

Baron-Cohen, S., Harrison, J., Goldstein, L. H., & Wyke, M. (1993). Coloured speech perception: Is synaesthesia what happens when modularity breaks down?. Perception, 22, 419-426.

6.

Beddor, P. S. (1993). The perception of nasal vowels. Nasals, Nasalization, and the Velum, 5, 171-196.

7.

Beddor, P. S., Krakow, R. A., & Goldstein, L. M. (1986). Perceptual constraints and phonological change: A study of nasal vowel height. Phonology Yearbook, 3, 197-217

8.

Beeli, G., Esslen, M., & Jäncke, L. (2008). Time course of neural activity correlated with colored-hearing synesthesia. Cerebral Cortex, 18, 379-385.

9.

Bien, N., ten Oever, S., Goebel, R. & Sack, A. T. (2012). The sound of size: Crossmodal binding in pitch-size synesthesia:A combined TMS, EEG and psychophysics study. NeuroImage, 59, 663-672.

10.

Bremner, A. J., Caparos, S., Davidoff, J., de Fockert, J., Linnell, K. J., & Spence, C. (2013). “Bouba” and “Kiki” in Namibia? A remote culture make similar shape-sound matches, but different shape-taste matches to Westerners. Cognition, 126, 165-172.

11.

Browman, C. P., & Goldstein, L. M. (1986). Towards an articulatory phonology. Phonology Yearbook, 3, 219-252.

12.

Browman, C. P., & Goldstein, L. M. (1988). Some notes on syllable structure in articulatory phonology. Phonetica, 45, 140-155.

13.

Browman, C. P., & Goldstein, L. (1990). Representation and reality: Physical systems and phonological structure. Journal of Phonetics, 18(3), 411-424.

14.

Cooke, T., Jäkel, F., Wallraven, C., & Bülthoff, H. H. (2007). Multimodal similarity and categorization of novel, three-dimensional objects. Neuropsychologia. 45, 484-495.

15.

Cuskley, C., & Kirby, S. (2013). Synaesthesia, cross-modality, and language evolution. In J. Simner & E. M. Hubbard (Eds), Oxford handbook of synaesthesia (pp. 869-899). Oxford: Oxford University Press.

16.

Cuskley, C., Simner, J. & Kirby, S. (2017). Phonological and orthographic influences in the bouba-kiki effect. Psychological Research, 81, 119-130.

17.

D'Andrade, R., & Egan, M. (1974). The colors of emotion. American ethnologist, 1, 49-63.

18.

De Klerk, V., & Bosch, B. (1997). The sound patterns of English nicknames. Language Sciences, 19, 289-301.

19.

Dixon, M. J., Smilek, D., Duffy, P. L., Zanna, M. P., &Merikle, P. M. (2006). The role of meaning in grapheme-colour synaesthesia. Cortex, 42, 243-252.

20.

Eagleman, D. M., Kagan, A. D., Nelson, S. S., Sagaram, D., &Sarma, A. K. (2007). A standardized test battery for the study of synesthesia. Journal of Neuroscience Methods, 159, 139-145.

21.

Gallace, A., & Spence, C. (2006). Multisensory synesthetic interactions in the speeded classification of visual size. Perception & Psychophysics, 68, 1191-1203.

22.

Hinton, L., Nichols, J., & Ohala, J. J. (2006). Sound symbolism. Cambridge: Cambridge University Press.

23.

Hirata, S., Ukita, J., & Kita, S. (2011). Implicit phonetic symbolism in voicing of consonants and visual lightness using Garner's speeded classification task. Perceptual and Motor Skills, 113(3), 929-940.

24.

Imai, M., & Kita, S. (2014). The sound symbolism bootstrapping hypothesis for language acquisition and language evolution. Philosophical Transactions of the Royal Society B, 369. 20130298.

25.

Imai, M., Kita, S., Nagumo, M., & Okada, H. (2008). Sound symbolism facilitates early verb learning. Cognition, 109, 54-65.

26.

Johnson, R. C., Suzuki, N. S., & Olds, W. K. (1964). Phonetic symbolism in an artificial language. The Journal of Abnormal and Social Psychology, 69(2), 233.

27.

Karwoski, T. F., & Odbert, H. S. (1938). Color-music. Psychological Monographs, 50(2, Whole No. 222).

28.

Kim, H.-W., Nam, H., & Kim, C.-Y. (2018). [i] is lighter and more greenish than [o]: Intrinsic association between vowel sounds and colors. Multisensory Research, 31, 419-437.

29.

Kim, H.-W., Nam, H., & Kim, C.-Y. (2016). Audiovisual association between consonants and colors in non-synesthetes. Journal of Vision, 16(12), 469-469.

30.

Köhler, W. (1929). Gestalt Psychology. New York, NY:Liveright.

31.

Kwak, Y., & Kim, C.-Y. (2018). Cross-modal correspondence between acoustic feature and shape. The Korean Journal of Cognitive and Biological Psychology, 30, 269-275.

32.

Lee Masson, H., Bulthé, J., Op de Beeck, H. P., & Wallraven, C. (2016). Visual and haptic shape processing in the human brain: Unisensory processing, multisensory convergence, and top-down influences. Cerebral Cortex, 26, 3402-3412.

33.

Marks, L. E. (1974). On associations of light and sound: The mediation of brightness, pitch, and loudness. The American Journal of Psychology, 87, 173-188.

34.

Maurer, D., Pathman, T., & Mondloch, C. J. (2006). The shape of boubas: Sound-shape correspondences in toddlers and adults. Developmental Science, 9, 316-322.

35.

Moos, A., Smith, R., Miller, S. R., & Simmons, D. R. (2014). Cross-modal associations in synaesthesia: Vowel colours in the ear of the beholder. i-Perception, 5(2), 132-142.

36.

Nam, H., Goldstein, L., Saltzman, E., & Byrd, D. (2004). TADA: An enhanced, portable Task Dynamics model in MATLAB. The Journal of the Acoustical Society of America, 115, 2430-2430.

37.

Newman, S. S. (1933). Further experiments in phonetic symbolism. The American Journal of Psychology, 45, 53-75.

38.

Nunn, J. A., Gregory, L. J., Brammer, M., Williams, S. C. R., Parslow, D. M., Morgan, M. J., . . . Gray, J. A. (2002). Functional magnetic resonance imaging of synesthesia:Activation of V4/V8 by spoken words. Nature Neuroscience, 5, 371-375.

39.

Ohala, J. J. (1986). Phonological evidence for top-down processing in speech perception. In J. S. Perkell & D. H. Klatt (Eds.), Invariance and variability in speech processes (pp. 386-397). Hillsdale, NJ: Lawrence Erlbaum.

40.

Paulesu, E., Harrison, J., Baron-Cohen, S., Watson, J. D., Goldstein, L., Heather, J., . . . Frith, C. D. (1995). The physiology of coloured hearing A PET activation study of colour-word synaesthesia. Brain, 118, 661-676.

41.

Palmer, S. E., Schloss, K. B., Xu, Z., & Prado-León, L. R. (2013). Music-color associations are mediated by emotion. Proceedings of the National Academy of Sciences, 110, 8836-8841.

42.

Parise, C. V., & Spence, C. (2009). ‘When birds of a feather flock together’: Synesthetic correspondences modulate audiovisual integration in nozn-synesthetes. PLoS One, 4(5), e5664.

43.

Peña, M., Mehler, J., & Nespor, M. (2011). The role of audiovisual processing in early conceptual development. Psychological Science, 22, 1419-1421.

44.

Pogacar, R., Plant, E., Rosulek, L. F., & Kouril, M. (2015). Sounds good: Phonetic sound patterns in top brand names. Marketing Letters, 26, 549-563.

45.

Ramachandran, V. S., & Hubbard, E. M. (2001). Synaesthesia--A window into perception, thought and language. Journal of Consciousness Studies, 8(12), 3-34.

46.

Rich, A. N., Bradshaw, J. L., & Mattingley, J. B. (2005). A systematic, large-scale study of synaesthesia: Implications for the role of early experience in lexical-colour associations. Cognition, 98, 53-84.

47.

Rubin, P. E., Saltzman, E., Goldstein, L. M., McGowan, R. S., Tiede, M. K., & Browman, C. P. (1996). CASY and extensions to the task-dynamic model. In Proceedings of the 1st ESCA ETRW on speech production modeling and 4th speech production seminar. Autrans.

48.

Saltzman, E. L., & Munhall, K. G. (1989). A dynamical approach to gestural patterning in speech production. Ecological Psychology, 1, 333-382.

49.

Sapir, E. (1929). A study in phonetic symbolism. Journal of Experimental Psychology, 12, 225-239.

50.

Schloss, I. (1981). Chickens and pickles. Journal of Advertising Research, 21, 47-49.

51.

Shepard, R. N. (1987). Toward a universal law of generalization for psychological science. Science, 237, 1317-1323.

52.

Shin, E. H., & Kim, C. Y. (2014). Both “나” and “な” are yellow: Cross-linguistic investigation in search of the determinants of synesthetic color. Neuropsychologia, 65, 25-36.

53.

Shinohara, K., & Kawahara, S. (2010). A cross-linguistic study of sound symbolism: The images of size. Berkeley Linguistic Society, 36(1), 396-410.

54.

Simner, J., Ward, J., Lanz, M., Jansari, A., Noonan, K., Glover, L., & Oakley, D. A. (2005). Non-random associations of graphemes to colours in synaesthetic and non-synaesthetic populations. Cognitive Neuropsychology, 22, 1069-1085.

55.

Sučević, J., Savić, A. M., Popović, M. B., Styles, S. J., &Ković, V. (2015). Balloons and bavoons versus spikes and shikes: ERPs reveal shared neural processes for shape-sound-meaning congruence in words, and shape-sound congruence in pseudowords. Brain and Language, 145, 11-22.

56.

Wrembel, M. (2009). On hearing colours-Cross-modal associations in vowel perception in a non-synaesthetic population. Poznań Studies in Contemporary Linguistics, 45, 595-612.

57.

Valdez, P., & Mehrabian, A. (1994). Effects of color on emotions. Journal of Experimental Psychology: General, 123, 394-409.

The Korean Journal of Cognitive and Biological Psychology