Collaboration
When Technology and Art Collide 2021-2022 Arts@ITRI Residency Project
ITRI’s research primarily focuses on technology development and application for the industry. Since the inception of Arts@ITRI Residency Project in 2019, however, ITRI has been delving into the role that technology can play in contemporary art. The 2021-2022 edition of the project, marking its third installment, has yielded impressive results, including the Mirage Replicas, Asymmetric Utopia, Whisper, Flowing Crowd with Musical Notes, (Aural)Embodiment, and War Awareness from Nagasaki 1945. Many of these multimedia artworks have been invited to participate in international digital art exhibitions, such as the 2023 Ars Electronica Festival in Austria, the 2023 Taiwan International Light Festival (TILF), the 2022 Taoyuan Art x Technology Festival, and the 2022 Nuit Blanche Taipei.
The artist and ITRI engineers worked on the Mirage Replicas’ dual-sided sensing transparent display to optimize the imaging effect based on the dancer’s location.
Hsiang-Lan Shih, Manager of the Service Systems Technology Center at ITRI, commented on the relationship between technology and art, stating, “Technology and art have often been viewed as separate entities. However, if we perceive them as an integrated concept, such as ‘techno-art,’ the two cultures can complement each other rather than one simply supporting or embellishing the other.” To foster a strong interdisciplinary partnership, artists and engineers, with their distinct working cultures, must collaborate and develop unique processes that enable artistic expression while optimizing compromises. As one artist aptly described it, this approach involves finding a delicate balance. Engaging in this project has encouraged ITRI engineers, accustomed to well-defined tasks and measurable objectives, to exhibit a high level of flexibility in rethinking technologies and co-creating interactive applications that prioritize the user experience.
To explore the impact of technology across generations, Yen-Tzu Chang collaborated with several ITRI laboratories specializing in intelligent robotics and transparent display virtual reality to create an intelligent multi-directional sensing display self-driving car, giving rise to the Mirage Replicas. The innovative vehicle seamlessly integrates the interaction between the dancer, audience, and device by capturing the dancer’s movements and tracking the gaze of the audience, while the transparent display projects virtual imagery of the dancer’s pre-recorded movements using cyber-physical fusion technology. Furthermore, engineers have developed a dual-sided sensing transparent display and streamlined the machine’s body to meet the visual aesthetic demands of the project.
The dancer rehearsed the immersive performance of ‘the Mirage Replicas’ that was inspired by Michel Foucault's concept of heterotopia.
Yung-An Wang’s Asymmetric Utopia re-imagines touch perception through a fictional character called the “Quantum Elemental,” who lacks pain sensation. To enable the audience to converse and interact with the Quantum Elemental, engineers with expertise in voice synthesis and recognition utilized a deep learning framework for bioacoustics to facilitate the training of a synthetic voice based on the artist’s own voice. The application of convolutional neural networks and attention models played a crucial role in swiftly achieving voice training for Wang’s creation.
In the last part of ‘Asymmetric Utopia’, the audience was invited to visit a massage provider to conclude their adventure with the guiding voice of the Quantum Elemental.
The visitor would experience the sense of surveillance upon entering the interactive installation ‘Whisper’.
Ning Tsai’s Whisper is an interactive theatrical installation that merges virtual and real sensory experiences. With virtual reality and fusion interactive technology experts, Tsai utilizes numerous mirrored panels, gaze-tracking sensors, and individual speakers to explore the relationship between gaze and discipline. As visitors enter the panoptic device, AI-generated faces stare or glance in their direction regardless of their movements; simultaneously, fragmented conversational whispers or the chaotic sounds of a crowd resonate in the background, their intensity varying based on the user’s proximity to the installation, constructing power dynamics through auditory and visual senses.
Flowing Crowd with Musical Notes is Siou-Ming Wu’s attempt to unravel the question, “What sound would emerge from the interweaving relationships of crowds in a space?” Leveraging AI image recognition technology and image processing, the system captures and converts individual images of people within the frame into dynamic data. The distance, speed, and duration between individuals are mapped onto musical staffs and notes, resulting in an improvisational performance of urban crowds.
The roaming crowd of people collectively formed an impromptu that encapsulated the city’s pulsating essence.
The movement of the dancer corresponds to the 8-channel surround sound designed by the space of the exhibition site, creating an interactive sound effect.
Olifa Hsieh’s (Aural)Embodiment showcases digital bodies constructing a virtual space through digital sounds. The technology centers around the creation of the dancer’s skeletal nodes that enable the dancer to interact with specific areas of the projected image to generate sounds. Through intelligent image analysis, the dancer’s movements are translated into virtual reality without the need for handheld controllers. With only a head-mounted display, the dancer becomes fully immersed in the virtual world.
Through the historical context of the atomic bombing in Nagasaki, Yi-Hua Wu’s War Awareness from Nagasaki 1945 sheds light on how people perceive mediated warfare. By projecting images captured by cameras on mechanical arms onto a transparent display film, Wu raises poignant questions on postmodern warfare. Engineers collaborated closely with Wu, imparting their expertise in autonomous vehicles and facilitating the integration of vehicles, imagery, scripts, and sound into a cohesive interface.
The contrasts between the robotic arm (covered with cloth on the left, it represents the presence of media in war), photos of Nagasaki in ruins projected on the screen, and ironic statement prompt reflection on mediated warfare.
To learn more about these artists and their artworks, feel free to visit the website of Arts@ITRI Residency Project.