A LEGENDAGEM SOB A ÓTICA DA SEMÂNTICA DE FRAMES: UMA ANÁLISE DE CONTRASTE SEMÂNTICO MULTIMODAL PORTUGUÊS-FRANCÊS EM OBRA AUDIOVISUAL

Raquel Figueiredo Roza; Maucha Andrade Gamonal; Adriana Silvina Pagano

doi:10.70860/ufnt.entreletras.e19563

Authors

Raquel Figueiredo Roza Federal University of Minas Gerais https://orcid.org/0009-0008-2306-3238
Maucha Andrade Gamonal Federal University of Minas Gerais https://orcid.org/0000-0002-5810-3913
Adriana Silvina Pagano Federal University of Minas Gerais https://orcid.org/0000-0002-3150-3503

DOI:

https://doi.org/10.70860/ufnt.entreletras.e19563

Keywords:

Frame Semantics, audiovisual translation, subtitle, multimodality, semantic similarity

Abstract

This study analyzes meaning construction in the subtitling from Brazilian Portuguese to French in the short film “Eu Não Quero Voltar Sozinho”, based on Frame Semantics. The research identifies the semantic similarity between audio and subtitles and explores how textual and visual information interact in interlingual translation. It adopts the multimodal annotation methodology of FrameNet Brasil, adapted to integrate visual elements. The results highlight how translational choices are expressed through frames and suggest ways to assess audiovisual translation. The study contributes to the multimodal dataset ReINVenta, aimed at developing frame-based artificial intelligence systems.

Downloads

Download data is not yet available.

Author Biographies

Raquel Figueiredo Roza, Federal University of Minas Gerais

Bacharel em Tradução com Ênfase no Português-Francês pela Universidade Federal de Minas Gerais (UFMG).

Maucha Andrade Gamonal, Federal University of Minas Gerais

Doutora em Linguística pela Universidade Federal de Juiz de Fora (UFJF), Juiz de Fora, MG – Brasil. Residente de pós-doutorado na UFJF.

Adriana Silvina Pagano, Federal University of Minas Gerais

Doutora em Letras pela Universidade Federal de Minas Gerais (UFMG). Professora Titular em Linguística Aplicada na UFMG.

References

BATEMAN, Jhon; WILDFEUER, Janina; HIIPPALA, Tuomo. Introduction: the challenge of multimodality. In. BATEMAN, Jhon, WILDFEUER, Janina., HIIPPALA, Tuomo. Multimodality: Foundations, Research and Analysis a Problem-Oriented Introduction. erlin/Boston: Walter de Gruyter GmbH, 2017. p. 7-21.

BELCAVELLO, Frederico; VIRIDIANO, Marcelo; COSTA, Alexandre Diniz da; MATOS, Ely Edison da Silva; TORRENT, Tiago Timponi. Frame-Based Annotation of Multimodal Corpora: Tracking (A)Synchronies in Meaning Construction. Proceedings of the International FrameNet Workshop 2020: Towards a Global, Multilingual FrameNet, Marseille, p. 23-30, mai., 2020. Disponível em: https://aclanthology.org/2020.framenet-1.4/. Acesso em: 07 jan. 2025.

BELCAVELLO, Frederico; VIRIDIANO, Marcelo; MATOS, Ely; TORRENT, Tiago Timponi. Charon: A FrameNet Annotation Tool for Multimodal Corpora. Proceedings of the 16th Linguistic Annotation Workshop (LAW-XVI) within LREC2022. Marseille, p. 91-96, jun. 2022, p. 91-96. Disponível em: https://aclanthology.org/2022.law-1.11/. Acesso em: 08 jan. 2025.

CINTAS, Jorge Díaz; REMAEL, Aline. Reconceptualizing subtitling. In: CINTAS, Jorge Díaz; REMAEL, Aline. Subtitling: Concepts and Practices. Oxon/New York: Routledge, 2021a. p. 1-31.

CINTAS, Jorge Díaz; REMAEL, Aline. The semiotics of subtitling. In: CINTAS, Jorge Díaz; REMAEL, Aline. Subtitling: Concepts and Practices. Oxon/New York: Routledge, 2021b. p. 91-117.

CINTAS, Jorge Díaz; REMAEL, Aline. Spatial and temporal features. In: CINTAS, Jorge Díaz; REMAEL, Aline. Subtitling: Concepts and Practices. Oxon/New York: Routledge, 2021c. p. 91-117.

CINTAS, Jorge Díaz; REMAEL, Aline. The Linguistic of Subtitling. In: CINTAS, Jorge Díaz; REMAEL, Aline. Subtitling: Concepts and Practices. Oxon/New York: Routledge, 2021d. p. 146-177.

CRESTANI, Fabio. Application of spreading activation techniques in information retrieval. Artificial Intelligence Review, [S. l.], v. 11, p. 453-482, 1997. DOI: https://doi.org/10.1023/A:1006569829653. Disponível em: https://link.springer.com/article/10.1023/A:1006569829653#article-info. Acesso em: 08 jan. 2025.

CZULO, Oliver. Aspects of a primacy of frame model of translation. In: CZULO, Oliver; HANSEN-SCHIRRA, Silva; HOFMANN, Sascha. Empirical modelling of translation and interpreting, Berlin: Language Science Press, 2017, p. 465-490.

DORNELAS, Letícia Grossi; GAMONAL, Maucha Andrade; PAGANO, Adriana Silvina. Análise semântica de audiodescrição em curta-metragem: uma abordagem multimodal a partir da Semântica de Frames. Domínios de Lingu@gem, Uberlândia, v. 18, p. 1-30, dez. 2024. DOI: https://doi.org/10.14393/DLv18a2024-66. Disponível em: https://seer.ufu.br/index.php/dominiosdelinguagem/article/view/75272. Acesso em: 08 jan. 2025.

FILLMORE, Charles J.; Semântica de Frames. Cadernos de Tradução, Porto Alegre, n. 25, p. 25-54, jul-dez., 2009.

HALVERSON, Sandra L. Implications of cognitive Linguistics for Translation Studies. In: ROJO, Ana; IBARRETXE-ANTUÑANO, Iraide. Cognitive Linguistics and Translation: advances in some theoretical models and applications. Berlin/Boston: Walter de Gruyter GmbH, 2013, p. 33-73.

HOLMES, James S. The Name and Nature of Translation Studies. In: VENUTI, Lawrence. The translation studies reader, London/New York: Routledge, 2000. p.172-185.

JAKOBSEN, Arnt; MAIA. Usando o Translog para registrar os acionamentos de teclas e mouse e rastrear os movimentos oculares de tradutores. Revista Graphos, João Pessoa, Edição Especial, p. 99-121, 2018. Disponível em: https://periodicos.ufpb.br/index.php/graphos/article/view/40075. Acessso 08 jan. 2025.

KRESS, Gunther. Mode. In: KRESS, Gunther. Multimodality: a social semiotic approach to contemporary communication. Oxon/New York: Routledge, 2010. p. 79-102.

ROJO, Ana; IBARRETXE-ANTUÑANO, Iraide. Cognitive Linguistics and Translation Studies: Past, present and future. In: ROJO, Ana; IBARRETXE-ANTUÑANO, Iraide. Cognitive Linguistics and Translation: advances in some theoretical models and applications. Berlin/Boston: Walter de Gruyter GmbH, 2013, p. 3-30.

RUPPENHOFER, Josef; ELLSWORTH, Michael; PETRUCK, Miriam; JOHNSON, Christopher; SCHEFFCZYK, Jan. FrameNet II: Extended Theory and Practice. 2016.

SALOMÃO, Maria Margarida Martins. FrameNet Brasil: um trabalho em progresso. Calidoscópio, São Leopoldo, v. 7, n. 3, p. 171-182, 2009. Disponível em: https://www.redalyc.org/articulo.oa?id=571561888002. Acesso em: 02 jan. 2025.

SAMAGAIO, Mairon Morelli. A Permanência Semântica na Tradução Audiovisual: uma análise da legendagem pela ótica da Semântica de Frames. 2023. 184f. Dissertação (Metrado em Letras) – Universidade Federal de Juiz de Fora, Juiz de Fora, 2023. Disponível em: https://repositorio.ufjf.br/jspui/handle/ufjf/16136. Acesso em: 08 jan. 2025.

VIRIDIANO, Marcelo; TORRENT, Tiago Timponi; CZULO, Oliver; MATOS, ALMEIDA, Arthur Lorenzi; MATOS, Ely Edison da Silva; BELCAVELLO, Frederico. The Case for Perspective in Multimodal Datasets. Proceedings of the 1st Workshop on Perspectivist Approaches to NLP. NLPerspectives. Marseille, p. 108-116, jun., 2022. DOI: https://doi.org/10.48550/arXiv.2205.10902.

TORRENT, Tiago Timponi; ELLSWORTH, Michael J. Behind the Labels: Criteria for Defining Analytical Categories in FrameNet Brasil. Veredas – Revista de Estudos Linguísticos, Juiz de Fora, v. 17, p. 44-65, 2013. Disponível em: https://periodicos.ufjf.br/index.php/veredas/article/view/25403. Acesso em: 08 jan. 2025.

TORRENT, Tiago; MATOS, Ely Edison da Silva Matos; BELCAVELLO, Frederico. VIRIDIANO, Marcelo; GAMONAL, Maucha, Andrade; COSTA, Alexandre Diniz da Costa; MARIM, Mateus Coutinho. Representing context in FrameNet: a multidimensional, multimodal approach. Front Psychol., [S. l.], v. 13, e838441, 2022. DOI: https://doi.org/10.3389/fpsyg.2022.838441. Disponível em: https://pubmed.ncbi.nlm.nih.gov/35444591/. Acesso em: 08 jan. 2025

WEBTOOL 4,02 [FNBr/server4/webtool402]. FrameNet Brasil. Disponível em: https://webtool.framenetbr.ufjf.br/index.php/webtool/annotation/main/annotation/628286/0/c. Acesso em: 08 jan. 2024.