Multimodal Means of Expressing Emotions in Interviews with Latin Americans about War Advertising: Experience of Multimodal Annotation

The article is devoted to the problem of identifying verbal and non-verbal means of expressing emotions in an oral interview about military advertising. The study is relevant due to the increased attention of linguists to the emotional processes in language and speech, the topicality of military discourse, and the interdisciplinary approach to the study of the linguistic nature of emotions by means of interviewing and multichannel annotation. What makes the work novel is the lack of research on emotional behavior of Spanish speakers in their discussing military advertising. The goal of the paper is to identify verbal and non-verbal means of expressing emotions in oral Spanish-language discourse while discussing military advertising. The material for the work incorporates video recordings of a structured interview with respondents from Latin America that were required to watch a commercial for the Colombian Armed Forces and answer some questions related to the content of the video and the occupation of the soldier. Methodological tools were an interview and a method of multichannel text annotation in ELAN. Multichannel annotation was carried out by means of marking five vocal and non-vocal tiers –“Verbal component”, “Gestures”, “Eye movement”, “Facial expressions”, “Intonation” – with subsequent structuring of the obtained data. Thus, it has been found that the video and military service evoke predominantly negative emotions among respondents – indignation, anxiety, surprise and disappointment. Negative emotions towards the video (disappointment, surprise and indignation) at the verbal level were expressed by lexical units with the common meaning “simplicity/plainness”, metaphorical comparisons with the film, rhetorical questions and negative indefinite pronouns – all emphasizing the lack of content and excessive romanticization of the military sphere, also enhanced by various manual illustrator gestures to express negative emotions and focus on certain semantic segments. The military profession was described through lexical units with the common meaning “risk/danger” and “injury/death”; respondents apparently experienced indignation and anxiety; a reference was made to a precedent situation – the current military-political conflict between Russia and Ukraine represented on a non-verbal level by various illustrator and regulator gestures. The interviewees displayed a feeling of discomfort through adapter gestures, facial movements and intonation, as well as a reluctance to maintain visual contact with the interviewer. Multichannel annotation allowed us to conclude that non-verbal means of expressing emotions can change the connotative meaning of the verbal component of a statement, indicate its untruthfulness and a desire of the respondent to hide their real attitude to the subject of discussion. The prospect of the study is seen in attracting respondents of other age and social groups.

1. Abrarova Z. F., Abrarov I. I. Social Advertising in Russia: Formaation of Positive Social Attitudes. Eurasian Legal Journal, 2018, no. 10 (125), pp. 506–508. (in Russ.)

2. Arutyunova N. D. Types of Linguistic Meanings. Evaluation. Event. Fact. Moscow, Nauka Publ., 1988, 336 p. (in Russ.)

3. Arutyunova N. D. Evaluation in the mechanisms of life and language. Language and the Human World. Moscow: Languages of the Russian Culture Publ, 1999, pp. 130–272. (in Russ.)

4. Apresyan Yu. D. Selected Works. Vol. 2., Integral Description of Language and System Lexicography. Moscow, Languages of Russian Culture, 1995, 767 p. (in Russ.)

5. Babenko L. G. Lexical Means of Denoting Emotions in the Russian Language. Sverdlovsk: Publ. of Ural university, 1989, 184 p.

6. Belyanin A. B. Social Advertising as a communicative management resourse: Abstract of the thesis. dis. … cand. soc. sciences. Moscow, 2007, 28 p. (in Russ.)

7. Wierzbicka A. «Sadness» and «Anger» in the Russian Language: the Non-Universality of the Socalled «Basic Human Emotions». Comparison of Cultures Through Vocabulary and Grammar. M.: Languages of Slavic Culture, 2001, pp. 15–43.

8. Vol’f E. M. Functional Semantics of Evaluation. Linguistic Legacy of the 20th Century. Moscow, Editorial URSS Publ., 2002, 228 p. (in Russ.)

9. Gornostaeva Yu. A., Kolmogorova P. A. Consolation Gestures as Non-Verbal Markers of Stress when Discussing an Acute Social Issue. Vestnik NSU. Series: Linguistics and Intercultural Communication, 2023, vol. 21, no. 1, pp. 30–53. (in Russ.).

10. Gornostaeva Yu. A., Kolmogorova P. A. Emotional Specifics of Verbal and Nonverbal Behavior When Discussing Body Positivity (based on interviews with informants from Russia). Perm University Herald. Russian and Foreign Philology, 2023, vol. 15, issue 2, pp. 24–39. (in Russ.).

11. Danilevskaya N. V. The Language of Russian Social Advertising of the New Times (Based on Texts Devoted to the Special Military Operation). Perm University Herald. Russian and Foreign Philology, 2023, vol. 15, issue 4, pp. 25–36. (in Russ.).

12. Ionova S. V., Shakhovskiy V. I. Anna Wierzbicka’s Linguocultural Theory of Emotions in the Deve­ lopment Dynamics. Russian Journal of Linguistics, 2018, vol. 22, no. 4, pp. 966–987. (in Russ.).

13. Kolmogorova, A. V., Gornostaeva, Yu. A. Discursive specificity of the emotional legitimization of the monarchy in the Spanish media. Bulletin of the Moscow State Pedagogical University. The series “Philology. Theory of language. Language education”, 2021, no. 3, pp.79–94. (in Russ.).

14. Kreidlin G.Ye. Nonverbal Acts and Touching Verbs. Word in Text and Dictionary: Collection of Articles for the 70th anniversary of Yu.D. Apresyan, 2000, pp. 109–121.

15. Lukyanova N. A. Expressive vocabulary of colloquial use (problems of semantics). Novosibirsk: Nauka, 1986. 230 p. (in Russ.)

16. Malikova A. V. Nonverbal markers of emotions for sentimental analysis of Russian-language Internet texts. Bulletin of the Northern (Arctic) Federal University. Series: Humanities and Social Sciences, 2020, no. 4, pp. 97–107. (in Russ.)

17. Sternin I. A., Prosovetsky D. Yu. Emotion and Evaluation in Semantic Words. World of Linguistics and Communication: Electronic Scientific Journal, 2018, no. 4, pp. 75–96. (in Russ.)

18. Telia V. N. Connotative Aspect of the Semantics of Nominative Units. Moscow, Nauka publ., 1986, 143 p. (in Russ.)

19. Firsova N. M. Reflection of National Features of Emotions of Spanish-speaking and English-spea­ king People in Verbal Means of the Communications. Russian Journal of Linguistics, 2004, no. 6, pp. 71–79. (in Russ.)

20. Firsova N. M. Reflection of Masculinity in Verbal and Nonverbal Means of Communication Among Spanish Speaking People. Russian Journal of Linguistics, 2006, no. 8, pp. 76–85. (in Russ.)

21. Khutyz I. P., Kolchevskaya V. A. Persuasiveness: the Specificity of the Phenomenon in Some Types of Institutional Discourse. Philological Sciences. Questions of theory and practice, 2018, no. 2(80), pp. 391–394. (in Russ.)

22. Shakhovskiy V. I. The Voice of Emotions in the Linguistic Circle of Homo Sentiens. Moscow, Stereotype, 2019, 144 p. (in Russ.)

23. Shakhovskiy V. I. Linguistic Theory of Emotions. Moscow, Gnozis Publ., 2008. 416 p. (in Russ.)

24. Shakhovskiy V. I. Human Emotions as an Object of the Study in Linguistics. Journal of Psycholinguistics, 2009, iss. 9, pp. 29–42. (in Russ.)

25. Shestopalova N. D. Oculesics as a meaning-forming element in the presentation of direct speech. Bulletin of Tver State University, 2015, no. 4, pp. 189–193. (in Russ.)

26. Yu Ye. D. On the Specifics of Nonverbal Behavior in Spanish-speaking Culture and the Linguistic Means of its Representation. iPolytech Journal, 2006, no. 4 (28), pp. 115–116. (in Russ.)

27. Yu Ye. D. Verbalization of Nonverbal Discourse Represented by Gestures and Tactile Communication in Spanish Literature (Fiction). iPolytech Journal, 2012, no. 10 (69), pp. 386–390. (in Russ.)

28. Ekman P., Friesen W. V. The Repertoire of nonverbal behavior: categories, origins, usage, and co­ ding. Semiotica, 1969, no. 1 (1), pp. 49–98.

29. Hjarvard S. The Mediatization of Society: a Theory of the Media as Agents of Social and Cultural Change. Nordicom Review, 2008, vol. 29, no. 2, pp. 105–134.

30. Jackson J. C. et al. Emotion semantics show both cultural variation and universal structure. Science, 2019, issue 366 (6472), pp. 1517–1522.

31. Jackson J. C., Watts J., List J.-M., Puryear C., Drabble R., Lindquist K. A. From Text to Thought: How Analyzing Language Can Advance Psychological Science. Perspectives of Psychological Science, 2022, iss. 17(3), pp. 805–826.

32. Lindquist K. A. Language and emotion: Introduction to the special issue. Affective Science, 2021, pp. 91–98.

33. Mehrabian A. Nonverbal communication. Nebraska symposium on motivation, 1971, vol. 19, pp. 107–161.

34. Mesquita B., Boiger M. Emotions in context: A sociodynamic model of emotions. Emotion Review, 2014, pp. 298–302.

35. Patel S., Scherer K., Sundberg J., Björkner E. Acoustic Markers of Emotions Based on Voice Physiology. Proceedings of the International Conference on Speech Prosody, International Speech Communications Association, 2010, vol. 100865, pp. 1–4.

36. Real Academia Española, Diccionario de la Lengua Española, Madrid, 1970. URL: https://www.rae. es/ (accessed on: 01.10.2023).

37. Trower P., Bryant B., Argyle M. Social Skills and Mental Health. London, Methuen, 1978, 320 p.

38. Wierzbicka A., Besemeres M. Emotion terms as a window on culture, social psychology and subjective experience. Language and Emotions: nominative and communicative aspects. Volgograd, 2009, pp. 966–987.