Photo: Yurika Takano

This work by contact Gonzo and yang02 was the continuation of an “untitled session” presented at Tokyo Arts and Space Hongo in 2019. In the performance, members of contact Gonzo are seemingly attacking each other as if they were fighting, all while an automatic guided vehicle (AGV) and a Segway modified to move on its own seek to “observe” and “explain” the scene. The performance was preceded by a work-in-progress version presented in Osaka in March and documented in a published report.*1

 *1　https://theatreforall.net/feature/feature-8568/

On the official website, two main points are raised pertaining to the work. One is cooperation with artificial intelligence (AI), and the other that the performance is being conducted as part of a festival called “TRANSLATION for ALL.”*2

*2　https://theatreforall.net/join/jactynogg-zontaanaco/

Let’s begin by addressing the first point, collaboration with AI.

The work makes use of a technology called image captioning, which automatically generates captions (or descriptive text) for images. The AI analyzes an image, detecting shapes and features in it, and interprets its content by recognizing objects in the image and their relationships. It then generates a caption using natural language processing methods such as deconstructing sentences to understand grammar and structure, understanding meaning from context, extracting specific information, and determining emotions and opinions contained in a sentence.

Silver lines on the floor of the venue served as tracks for the AGV, which can carry a load of up to 100 kilograms and sometimes moved with a performer on board, while the Segway had been modified to move autonomously without a human driver. Cameras mounted on each of the devices took still images every ten seconds or so, and AI-generated captions for those images were projected onto the venue’s walls. The AGV’s captions were read out in a male computer voice, the Segway’s in a female. Since audience members could be exposed to the machines’ cameras, contact Gonzo provided masks to those who wished to hide their faces.

Photo: Yurika Takano

contact Gonzo’s performances resemble a martial art, with the participants’ bodies colliding repeatedly with each other, and it isn’t clear whether any rules govern their movement. The heavy sound of body slams and the performers’ disordered breathing make one feel uneasy. Explaining the scene in words would seem very difficult. For the purposes of this work, the AI was described as playing the role of a translator that converts the performance into language. How did that turn out?

As in the image below, the AI-generated captions described something different from that happening in front of my eyes. But was it really “different”?

The performers made adjustments such as changing costumes and picking up instruments, as if fighting to see how the AI would interpret them while their performance. I guess they were trying to see if they could get it to produce an appropriate caption. But looking at the captions actually generated, one could vaguely make out the content of, and trends in, the data the AI had been trained on, and I felt like I was watching the show while chatting with a non-human intelligence—an interpretation more interesting than evaluating the AI’s abilities for translation or explanation. The AI appeared to pick up a certain sadness in the performers’ eyes, one I didn’t see.

Photo: Yurika Takano

Let’s move on to the second point, “TRANSLATION  for ALL.”

As stated on the project’s website, “There are all sorts of barriers on the road to delivering physical expression to audiences. The various works [presented] are taking up the challenge of overcoming [those barriers] through diverse techniques and opening up accessibility to ALL, meaning all kinds of people.”*3

 *3 TRANSLATION for ALL: https://theatreforall.net/translation-for-all/

The organizers added live commentary by a human after the work-in-progress performance, and at the dress rehearsal of this showing they consulted a monitoring group composed of visually impaired individuals. The commentary was provided by the dancer Akiyoshi Nita, who whispered into a microphone, as if bringing an explanation intended only for someone right next to or in front of him to the entire audience. Conveying the progress of the performance and the participants’ movements, Nita’s commentary accurately described the way these bodies were moving—such as how arms were bent—and their intricate interplay, all while infusing his words with the atmosphere of the scene and the rhythm of the performers. His rap-like delivery made for pleasant listening.

Photo: Yurika Takano

The AI “chatter” and Nita’s commentary were enjoyable as elements of the performance, but how did they further the goal of “opening up accessibility to all”? In the project handout, this was addressed as follows: “In an attempt to promote accessibility in the appreciation of artworks, we will attempt to ‘translate’ physical expression by incorporating, as elements of the artwork, verbal explanations of the performance by an AI and a human, as well as textual support through UD TALK [a communication support app developed in Japan] and subtitles.”

By including elements such as commentary, linguistic explanations, and UD TALK compatibility, the organizers had undoubtedly gone to great lengths to help the audience enjoy the work. However, I felt that including these “as elements of the artwork,” and making the work/performance accessible in itself, are matters that differ in kind.

Artistic expression focuses on issues related to the method and intent of an artwork, while accessibility emphasizes making that artwork approachable to anyone. Precisely because this performance was experimental, I felt that it could have involved people with a stake in accessibility from the outset, thereby enabling the organizers to separate and coordinate these aspects—and open up even greater artistic possibilities.

Let me end on a different note. During the intermission, the owners of Leggy_ (plant sales) and Ken-chan Curry, who had stalls at the venue, talked about how they are attracted to plants that have grown into unwanted shapes and collect them by going from one gardening store to the other. Similarly to these overgrown succulents, “jactynogg zontaanaco” appeared to reveal the appeal of AI that materializes in situations beyond the optimal. I look forward to seeing new attempts in this vein in the future.

Translated by Ilmari Saarinen