ITRI TODAY Spring issue 2024

Glimpse into Our AI Entertainment & Robotics Tech

Our AI display and entertainment innovations exhibited at CES 2024 included the AR Interactive Vehicle Display, a transparent display for sightseeing passengers in a moving vehicle to view and interact with augmented reality (AR) content regarding attractions they pass by; the Hyper-Realistic 3D Interactive Display, delivering 3D animations and live broadcasts with a digital avatar; and PetPet Cam, the world’s first Gen AI pet camera that accurately captures our furry friends’ cute moments and generates mashups for personal cherishing or easy sharing.

The AI robotics innovations ITRI presented are the Detachable Robot Joint System, a highly flexible robot joint module that allows rapid assembly into a plug-and-play robotic arm with customizable axes, and RoboTwin: Metaverse Smart Factory Simulation Platform for creating a realistic virtual reality (VR) simulation of a manufacturing facility.

AR Interactive Vehicle Display brings AR to sightseeing

The AR Interactive Vehicle Display allows tour bus passengers to view and interact with AR content regarding attractions along their route. When passengers spot a site of interest, the location name is presented in real time, following their line of sight. A simple touch on the screen provides detailed information, such as attraction features, local history, and geography. This versatile technology can also be applied to light rail systems, cable cars, cruise ships, and more.

This technology has expanded its reach through collaboration with leading display solution provider AUO on a ferry operating in the Port of Kaohsiung. AUO has taken the infotainment concept from the water to the road, transforming it into the Interactive Transparent Window. This innovation was named a CES 2024 Best of Innovation Honoree, showcasing ITRI’s commitment to empowering Taiwanese companies on the global stage.

Hyper-Realistic 3D Interactive Display interacts with your avatar

The Hyper-Realistic 3D Interactive Display is the world’s first interactive system that can interact with your digital avatar and deliver 3D animations and live broadcasts. Powered by generative AI (Gen AI), the system can carry out real-time image processing, including 3D models, image matting, expression, motion generation, single 2D to 3D imaging, and 3D virtual-sense fusion, making it ideal for creative advertising and marketing campaigns.

Key features include:

• Gen AI 2D to 3D imaging: Using advanced AI technology, the system swiftly converts 2D images into an intricate avatar in 3D.
• Smart virtual-real 3D image fusion: The avatar appears 3D to the naked eye and can maintain eye contact with viewers from various angles.
• AI expression and motion generation: Continuous expressions of the avatar can be synthesized from a single image and voice. Its movements are also lifelike and accurate because of the automatic skeleton binding technology.

PetPet Cam captures pets’ “cute events”

PetPet Cam utilizes Gen AI to train a deep neural network (DNN) model to accurately capture “cute events” of pets, such as yawning, belly showing, stretching, and playful interactions with family members. It automatically generates photo and video mashups that owners can effortlessly share on social media platforms.

The onboard edge AI works like an experienced photographer, reacting instantly to the pet’s actions and deftly controlling the shutter. PetPet Cam’s autonomous photo capturing reaches 95% accuracy for defined events, saving the owner from searching through hours of footage to find a specific frame.

PetPet Cam identifies individual pets, tracks their drinking and feeding frequency, and provides valuable insights for pet owners and healthcare professionals. Its versatile design allows users to set it up in the pet’s living area or use multiple cameras to cover various spaces.

Detachable Robot Joint System enables rapid robotic arm assembly

The Detachable Robot Joint System can be rapidly assembled into a plug-and-play robotic arm with a customizable number of axes for real-time motion commands. It enables the customized creation of robots with high payload, multiple axes, or wide-ranging operations, catering to the demand for flexible and small-batch production.

While traditional robotic-arm repairs can take days, this innovative module can have its axis replaced in just five minutes, significantly enhancing production efficiency while reducing costs, time, and manufacturing risks. Compared to industry-standard joint modules, the Detachable Robot Joint System is not only lighter but also boasts a superior load-to-weight ratio and the world’s highest torque-to-volume ratio.

RoboTwin brings virtual reality to factories

Through state-of-the-art AI and Sim2Real technologies, RoboTwin: Metaverse Smart Factory Simulation Platform creates a realistic VR simulation of a manufacturing facility based on real-world data. Engineers and operators wearing VR headsets can immerse themselves in the virtual factory, simulating production-line adjustments and performing tasks.

RoboTwin’s customizable applications include industrial metaverse experiences, human-robot interactive simulation, and various applications for multi-agent collaboration, remote troubleshooting, and maintenance via teleoperation.

Leading Taiwanese manufacturers, China Steel Machinery Corporation (CSMC) and TECO Electric & Machinery Co., Ltd (TECO), have utilized RoboTwin to create virtual factories, enhancing efficiency in their production processes. RoboTwin also holds significant application potential in industries such as semiconductors, machining, and manufacturing.