Search

A Study on the Differences in the Integration of Different Types of Gestures and Language Based on the Modality-Specific Simulation

Author(s)

Hukai Xu

Affiliation(s)

School of Innovation and Development, Central University of Finance and Economics, Beijing, China

Abstract

The integration of gestures and language is a core topic in the study of multimodal communication. From the theoretical perspective of modality-specific simulation, this paper classifies gestures into four types: indicative, metaphorical, rhythmic, and pictographic. It systematically compares the differences between gesture and language integration in terms of semantic integration depth, temporal process, neural mechanism, and cognitive load, and provides a unified explanation using the modality compatibility hypothesis, the extended dual encoding theory, and the predictive coding framework. Finally, it explores the practical value of this in education and clinical rehabilitation. The research shows that the differences in the integration of different types of gestures are rooted in the different attributes and processing pathways of the modality-specific simulations they activate.

Keywords

Modal Specificity Simulation; Gesture Type; Language Integration; Embodied Cognition; Dual-Stream Model; Modal Compatibility

References

[1] Barsalou, L. W. (2008). Grounded cognition. Annu. Rev. Psychol., 59(1), 617-645. [2] Mahon, B. Z., & Caramazza, A. (2008). A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. Journal of physiology-Paris, 102(1-3), 59-70. [3] Hickok, G., & Poeppel, D. (2007). The cortical organization of speech processing. Nature reviews neuroscience, 8(5), 393-402. [4] Alibali, M. W. (2005). Gesture in spatial cognition: Expressing, communicating, and thinking about spatial information. Spatial cognition and computation, 5(4), 307-331. [5] Gallese, V., & Lakoff, G. (2005). The brain’s concepts: the role of the sensorimotor system in reason and language. Cognitive Neuropsychology, 22(3), 455-479.