Consolation Gestures as Non-Verbal Markers of Stress when Discussing an Acute Social Issue

The article is devoted to the issue of identifying non-verbal stress markers in oral discourse on an acute social topic. The relevance of the research is due to: 1) the growing interest of linguists in the study of linguistic manifestations of emotions and the identification of markers of various psycho-emotional states with the subsequent possibility of developing algorithms for their automatic detection; 2) the severity of the problem of stress in modern society. The novelty of the study is predetermined by the fact that it is focused on a specific type of communicative behavior (discussion of an acute social issue) studied from the perspective of multimodal linguistics postulating an inextricable link between understanding and generating messages in natural language and body actions. The goal of the paper is to identify and describe non-verbal stress markers that we call consolation gestures, allowing us to identify this psychoemotional state in the course of spontaneous oral communication. The research material is represented by 4 hours of structured interview video recordings with 11 French and 11 Russian respondents (average age – 21) on the topic of radical body positivity. The leading research method is multimodal analysis, implemented using multi-layer markup in the ELAN program. In the process of marking, the following types of layers were used: gestures, eye movements, facial expressions, verbal component, intonation. The statistical analysis of frequency and distribution of annotations have revealed the following consolation gestures: 1) stroking different parts of the body; 2) scratching different parts of the body, 3) interacting with other objects, 4) facial markers. Consolation gestures are regarded by us as a kind of body-oriented repetitive behavior disorders, which allows us to differentiate them from gestures that are hesitation/silence fillers. Thus, the following constitutive features are determined: 1) repetition of the action, 2) focus on the body, 3) features conditioned by a sense of tension or anxiety rather than by preoccupation with appearance, 4) a subsequent sense of relief and emotional discharge. The prospect of further research is seen in the expansion of experimental material through interviews with representatives of other linguocultures, followed by interpreting the results in the cross-cultural perspective.

1. Atamanova, O. V. Investigation of the linguistic-statistical nature of stress as a synergetic phenomenon. International Research Journal. Philological sciences, 2018, no. 8 (74), pp. 122–125. (in Russ.)

2. Babenko, L. G. Lexical means of emotion designation in the Russian language. Sverdlovsk: Ural State University, 1989. 184 p. (in Russ.)

3. Balaganov, D. V. Influence of stress on the activity of a simultaneous interpreter. Philological sciences. Questions of theory and practice, 2018, no. 12(90), pt 1, pp. 74–78. (in Russ.)

4. Baranov, A. N. Word and gesture in linguistic examinations in cases of bribes: towards the semantics and pragmatics of “closed” discourses. Bulletin of Volgograd State University. Series 2: Linguistics, 2020, vol. 19, no. 1, pp. 64–76. (in Russ.)

5. Grif, M. G., Korol’kova, O. O., Manueva, Yu. S. Development of algorithmic and software for choosing the meaning of a polysemous word and homonym in the system of computer sign language translation of the Russian language based on a semantic model. Vestnik NSU. Series: Linguistics and Intercultural Communication, 2018, vol. 16, no. 3, pp. 32–44. (in Russ.)

6. Grishina, E. A. Gestures and pragmatic characteristics of utterance. Multimodal communication: theoretical and empirical research. Moscow: Buki-Vedi, 2014, pp. 25–47. (in Russ.)

7. Ioanesyan, E. R., Dronov, P. S. Natural language nominations of gestures with adaptors. Bulletin of the Udmurt University. The series “History and Philology”. 2020, vol. 30, iss. 6, pp. 959–967. (in Russ.)

8. Kendon, A. The study of gesture: Some observations on its history. Recherches Sémiotiques / Semiotic Inquiry, 1982, no. 2(1), pp. 45–62.

9. Kibrik, A. A. Multimodal linguistics. Cognitive research, 2010, iss. IV, pp. 134–152. (in Russ.)

10. Kibrik, A. A., Fedorova, O. V., Podlesskaya, V. I. Multichannel buildings: yesterday, today, tomorrow. Humanities readings of RSUH – 2017. Moscow: RSUH, 2018, pp. 499–511. (in Russ.)

11. Kolmogorova, A. V., Gornostaeva, Yu. A. Discursive specificity of the emotional legitimization of the monarchy in the Spanish media. Bulletin of the Moscow State Pedagogical University. The series “Philology. Theory of language. Language education”, 2021, no. 3. Pp. 79–94. (in Russ.)

12. Kolmogorova, A. V., Gornostaeva, Yu. A. Verbal markers of manipulation in English polarized political discourse: the experience of parametrization and automatic processing. Krasnoyarsk: Sib. feder. un-t, 2021. 188 p. (in Russ.)

13. Kreidlin, G. E. Non-verbal semiotics: body language and natural language. M.: New Literary Review, 2002. 592 p. (in Russ.)

14. Kudinova, E. S. Modern methods of speech-to-speech interaction research. Bulletin of the Moscow State Linguistic University. Humanities, 2020, iss. 1(830), pp. 122–133. (in Russ.)

15. Malikova, A. V. Nonverbal markers of emotions for sentimental analysis of Russian-language Internet texts. Bulletin of the Northern (Arctic) Federal University. Series: Humanities and Social Sciences, 2020, no. 4, pp. 97–107. (in Russ.)

16. Panicheva, P. V. Linguistic modeling of stress, well-being and dark personal characteristics based on the texts of Russian-speaking Facebook users. Structural and Applied Linguistics, 2015, no. 11, pp. 240–251. (in Russ.)

17. Suhova, N. V. Meanings of sign units: on the question of annotating head gestures. A.D. Krivonosov (ed.) Russian Public Relations-4: trends and drivers: a collection of works in honor of Professor L. V. Minaeva. St. Petersburg: Publishing House of St. Petersburg State University, 2017, pp. 82–

18. (in Russ.)

19. Fedorova, O. V. On Russian gesticulation from a linguistic point of view (to the publication of the monograph by E. A. Grishina). Questions of Linguistics, 2018, no. 5, pp. 114–123. (in Russ.)

20. Shahovskij, V. I. Emotions as an object of research in linguistics. Questions of psycholinguistics, 2009, no. 9, pp. 29–42. (in Russ.)

21. Shiller, A. V. The role of theories of embodied cognition in research and modeling of emotions. Philosophical Sciences, 2019, no. 62(5), pp. 124–138. (in Russ.)

22. Blache et al. Multimodal annotation of conversational data. The Fourth Linguistic Annotation Workshop. ACL, 2010. Pp. 186–191.

23. Brault, G. J. Kinesics and the Classroom: Some Typical French Gestures. The French Review, 1963, vol. 36, no. 4, pp. 374–382.

24. Brugman, H., Russel, A. Annotating Multimedia. Multi-modal resources with ELAN. Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04). Lisbon, 2004.

25. Calbris, G. The Semiotics of French Gestures. Indiana University Press, 1990. 260 p.

26. Damasio, A. R. Descarte’s error: emotion, reason, and the human brain. New York: Putnam Publishing, 1994.

27. Damasio, A. R. The somatic marker hypothesis and the possible functions of the prefrontal cortex. Philosophical Transaction: Biological Sciences, 1996, no. 351, pp. 1413–1420.

28. Efron, D. Gesture, Race and Culture: A Tentative Study of the Spatio-Temporal and Linguistic Aspects of the Gestural Behavior of Eastern Jews and Southern Conditions (Approaches to Semiotics). The Hague: Mouton & Co, 1972. 226 p.

29. Ekman, P., Friesen, W. V. The Repertoire of nonverbal behavior: categories, origins, usage, and coding. Semiotica, 1969, no. 1(1), pp. 49–98.

30. Ferré, G., Bertrand, R., Blache, Ph., Espesser, R., Rauzy, S. Intensive Gestures in French and Their Multimodal Correlates. Interspeech, 2007, no. 27-31, pp. 690–693.

31. Gullberg, M., Hendriks, H., Hickmann, M. Learning to talk and gesture about motion in French. First Language. Sage Publications. 2008, vol. 8(2), pp. 200–236. DOI 10.1177/0142723707088074

32. Kendon, A. The study of gesture: Some observations on its history. Recherches Sémiotiques/Semiotic Inquiry, 1982, no. 2(1), pp. 45b–62.

33. Kendon, A. Gesture: Visible action as utterance. Cambridge: Cambridge University Press, 2004. 400 p.

34. Litvinenko, A. O., Kibrik, A. A., Fedorova, O. V., Nikolaeva, J. V. Annotating hand movements in multichannel discourse: Gestures, adaptors and manual postures. Rossijskij zhurnal kognitivnoj nauki, 2018, no. 5(2), pp. 4–17.

35. McNeill, D. Hand and Mind: What Gestures Reveal about ‘Thought’. Chicago: Chicago University Press, 1992. 423 p.

36. Mehl, M. R. et al. Natural language indicators of differential gene regulation in the human immune system. Proceedings of the National Academy of Sciences, 2017, vol. 114, no. 47, pp. 12554– 12559.

37. Nikolaeva, J. V. Pragmatic gestures in Russian Retellings of “The peer stories”, 2017, no. 4 (2-3), pp. 6–12.

38. Phillips, K. A., Stein, D. J. Handbook on Obsessive-Compulsive and Related Disorders. The American Journal of Psychiatry, 2015, vol. 172, iss. 11, pp. 1164–1165.

39. Ruben, R. J. Sign language: its history and contribution to the understanding of biological nature of language. Acta Oto-laryngologica, 2005, no. 125, pp. 464–467.

40. Saslow, L. R. et al. Speaking under pressure: Low linguistic complexity is linked to high physiological and emotional stress reactivity. Psychophysiology, 2014, no. 51(3), pp. 257–266.

41. Wilson, R. A., Foglia, L. Embodied Cognition. The Stanford Encyclopedia of Philosophy (2017 Edition) [Online]. URL: https://plato.stanford.edu/archives/sum2017/entries/embodiedcognition (accessed on: 04.10.2022).