China's humanoid robots advance with enhanced emotional interaction

A humanoid robot winks at the World Robot Conference 2024 in Beijing, capital of China, Aug. 21, 2024. (Xinhua/Chen Zhonghao)

BEIJING, Aug. 26 (Xinhua) -- Wrapping sushi rice up with his nimble hands, a silver-haired "master chef" leaned over the sushi counter and focused on his craft with visible dedication. In front of him, the conveyor belt is overflowing with aromatic tuna sushi and delectable sushi rolls, taking visitors on an immersive journey to an authentic sushi bar nestled in a Tokyo neighborhood.

But in reality, the visitors were surrounded by legions of humanoid robots, including the "master chef" himself. These robots, with their strikingly lifelike appearances, featured clearly visible wrinkles on their faces and veins under their "skin," blurring the line between human and machine.

Such vivid scenes are no longer limited to flashy demo clips but are now part of a demonstration by EX-Robots, a Dalian-based leading company in China's robotics industry. Here, many of these humanoid robots have moved beyond sleek, futuristic designs to astonishingly human-like forms, fostering a more natural and relaxed atmosphere for human-robot interaction.

According to Li Boyang, co-founder and CEO of the company, the key lies in their enhanced facial expressions and the ability to foster an emotional connection with users.

"In terms of core technologies, our unique features include the robot's intelligent facial expressions and its dexterous hands. Our algorithms focus on multimodal large models that enhance emotional and perceptual capabilities, as well as intelligent decision-making and human-like interaction abilities, such as speech, facial expressions and gestures," Li said.

Li further highlighted that the company is committed to improving how robots understand and respond to human emotions, particularly during conversations and when reading aloud. This research is part of a broader effort to advance robots' cognitive abilities, specifically their skills in recognizing facial expressions and emotional cues.

Currently, EX-Robots produces between 400 and 500 units annually, primarily used in museums and educational institutions. However, the company anticipates that as production costs decrease, there will be broader applications for its robots in many fields.

Looking ahead, many industry insiders including Li believe humanoid robots with emotional interaction capabilities will play a larger role in sectors including healthcare and education, with potential applications in psychological counseling and health management.

China, a major global player in the humanoid robot sector and the world's largest market for robot applications, is seeing an increasing focus on deploying robots in scenarios that require emotional interaction. Companies including EX-Robots are at the forefront of this movement, pushing the boundaries of what robots can do.

A humanoid robot is pictured during the World Robot Conference 2024 in Beijing, capital of China, Aug. 21, 2024. (Xinhua/Ren Chao)

In addition to corporate advancements, academic research is also contributing to the field.

A research team led by Liu Xiaofeng, a professor at Hohai University in Jiangsu Province, has introduced a new algorithm for generating facial expressions on humanoid robots. Their findings, detailed in the July issue of IEEE Transactions on Robotics, present a novel approach to action unit (AU)-driven facial expression synthesis.

Thanks to advanced AI models, robots can now understand human language and the emotions conveyed through tone, and they can interact through speech. However, Liu noted that creating appropriate facial expressions while "speaking" remains a significant challenge for humanoid robots.

To address this challenge, according to Ni Rongrong, a co-author from Changzhou University, the team divided the nine motors on the humanoid robot's face into 17 AUs to enable richer expressions and smoother transitions through coordinated movements.

"Each unit is like a letter, and by combining different letters, we form words," Ni explained.

Liu envisions that as emotional interaction capabilities improve, these robots will increasingly be used in settings such as nursing homes, kindergartens, and special education schools, where both high emotional and intellectual engagement are valuable.

However, at the just-concluded 2024 World Robot Conference in Beijing, feedback from the audience has been mixed. Some visitors expressed enthusiasm for robotic companions, while others voiced concerns.

Standing by an array of humanoid robots mimicking human speech, with their cheeks fluctuating rhythmically, a visitor surnamed Lyu described the scene as "a little bit creepy." "I know the cause may be the uncanny valley effect, but I just can't imagine enjoying emotional interaction with them," she added.

Adding to the discussion, Ren Lei, a professor at Jilin University, said that current humanoid robots have yet to achieve a perfect blend of form and essence. "They only simulate human appearance but fall short in mimicking human essence. Their agility, dexterity, and precision cannot compare to those of humans. Due to these functional limitations, many humanoid robots still lack practical applications and are far from being fully implemented in real-world scenarios."