Last summer, a post on Shayla Reyes’ Facebook feed caught her attention. It was a picture of a friend in Japan posing with another person in their late ‘20s.
“The accompanying text read, ‘With my grandson’,” said Reyes, a master of media studies student in the Faculty of Information and Media Studies (FIMS).
“To my knowledge, my friend had never posted in English before and although I had not talked with him in 12 years, I know he does not have a grandson. I also knew his English proficiency is high enough for him to definitely understand what ‘with my grandson’ means,” she said.
Confused, Reyes dug a little deeper.
“That’s when I noticed the ‘see original’ button under the text,” she said. “When I clicked on it, it said, ‘With Mr. Tora’ (a person’s name) in Japanese.”
The post was the result of machine translation, which automatically adjusted the Japanese text into English.
Reyes, a Canadian Filipina who speaks English and Japanese, has been using Facebook since 2009 to stay in touch with relatives who speak Filipino, English and French, and her friends, who speak English and Japanese.
“I didn’t know Facebook was automatically translating the language for me, because my setting is ‘English,’” she said. “The ‘see original’ button is almost invisible until you notice it. If his text had read, ‘my brother,’ instead of ‘grandson,’ I would have believed it.”
Experiencing how Facebook’s automatic translation can distort the user’s intended meaning became the basis of her master’s thesis and independent research-creation project, Flip to See Original.
It is now on display on the second floor of the FIMS/Nursing building.
Reyes aims to draw attention “to the invisibility of Facebook’s machine translation (MT) and make visible how it is mediating communication involving bilingual and multilingual users.”
“I’m not sure people realize Facebook‘s MT is taking away the user’s choice of language and translating their texts without their knowledge,” she said. “Most of my friends didn’t.”
Reyes said bilingual and multilingual users are impacted the most, with different reasons dictating why they choose one language over another when communicating.
“This can depend on the topics they are discussing, whom they are talking to or simply the level of their language proficiency,” she said.
Sharing knowledge through artistic expression
Under the supervision of FIMS professors Sarah Smith and Anabel Quan-Hasse, and with a background in art and intermedia, Reyes chose to share the findings of her work through a research-creation project.
The approach combines creative and academic research practices and supports the development of knowledge and innovation through artistic expression, scholarly investigation and experimentation.
Smith said Reyes applied the principles of the method in an effective and engaging manner.
“Shayla’s project brings together critical engagement with scholarship on social media and translation with a creative approach, resulting in an engaging public-facing series of artworks that convey the issues that arise with Facebook’s machine translation,” she said.
“A key aspect of her project invites viewer engagement, which is essential to her argument that we need to build broader public awareness of how machine translation mediates communication on these platforms.”
‘Flipping’ for the facts
Reyes created a series of artworks designed to mimic Facebook posts, with the size and shape of each image resembling a smartphone screen – the most common way people engage with social media.
Each artwork consists of three layers of image and text, depicting an everyday scenario that focuses on Japanese culture.
The top layer shows Facebook’s machine translated text from Japanese to English, alongside an image that represents the text. The middle layer shows the machine translated text from English to Japanese, with an image that represents that text. The final, bottom layer shows the original English and Japanese texts accompanied by the original image.
“You have to flip two layers to see the original, just like you have to click to see the original on Facebook,” Reyes said. “I’m mimicking that experience.”
Shayla Reyes designed each artwork in her project to mimic how a bi- or multilingual Facebook user has to click through ‘layers’ to see the original text of a post. The top layer shows Facebook’s machine translated (MT) text from Japanese to English, alongside an image representing the text. The middle layer shows the MT text from English to Japanese, with accompanying image and the bottom layer shows the original English and Japanese texts accompanied by the original image. (Faculty of Information and Media Studies.)
Findings
Reyes found most translations from English to Japanese were accurate, but those from Japanese to English were mostly inaccurate.
“It could be because there are Japanese words very specific to Japanese culture,” she said. “Languages are strongly connected to culture and when posting in a specific language, the cultural nuances of that language are incorporated. When multilingual users convey the same messages in different languages, it may not be a direct translation.”
One striking example of this was the mistranslation of “Sakura,” the Japanese term for cherry blossom trees.
The Japanese phrase “I went to see the cherry blossoms. They are beautiful,” was translated by Facebook’s MT to read in English as “I came to see the birds. Bored.”
“That one surprised me the most and made me laugh,” Reyes said. “Cherry blossoms are one of the most popular tourist attractions to Japan. I wrote three similar sentences about cherry blossoms specifically including the word ‘Sakura,’ and all translations failed.”
Reyes is hoping to one day further her research by interviewing bi- and multilingual users about their experiences with Facebook’s MT.
For now, she hopes those viewing her exhibit will consider the role machine translation could be playing in their communication on Facebook and any assumptions they may have about its use.
Reyes’ exhibit is on display until the end of May on the second floor of the FIMS/Nursing building.
