Research team from China's Zhejiang University enables brain-controlled robotic arm to write Chinese characters for first time

A brain-computer interface research team from Zhejiang University in China has achieved a first by successfully enabling a robotic arm controlled by the mind through an invasive brain-computer interface to write Chinese characters.

The team's latest research achievement was released on Tuesday at the Second Affiliated Hospital Zhejiang University School of Medicine (SAHZU) in Hangzhou, capital of east China's Zhejiang Province.

During the release event, the research team operated a robotic arm that smoothly wrote Chinese characters meaning "Zhejiang University" and "Brain-Computer Interface" based on brain signal data previously extracted from a male patient surnamed Zhang.

A patient writes Chinese characters by controlling a robotic arm through an invasive brain-computer interface. (Photo courtesy of the Second Affiliated Hospital Zhejiang University School of Medicine)

"Some stroke survivors and patients suffering from amyotrophic lateral sclerosis (ALS) lose their writing and speaking abilities. But by extracting their brain signals to control external mechanical devices, they can write out what they want to say and communicate with the outside world. This is a major breakthrough," said Zhang Jianmin, director of the neurosurgery department of SAHZU.

The research team has been conducting pioneering interdisciplinary research into brain-computer interfaces since 2006.

In 2012, it achieved the breakthrough of decoding brain signals for precise hand gestures such as hooking, grasping, pinching, and gripping for the first time by implanting electrodes in the motor cortex of a monkey.

In 2014, the team completed the first domestic clinical trial of implanting electrodes into a patient's skull and successfully decoding brain signals for hand motions used in the rock-paper-scissors game.

In 2020, the research team successfully enabled a senior clinical volunteer to use an invasive brain-controlled robotic arm to complete tasks including drinking water, eating, and shaking hands. The experiment marked a first in China.

The team's latest focus is brain-controlled Chinese character writing. With this technology, patients only need to think about the normal writing process and the robotic arm can obtain information about the writing trajectory in their minds by decoding neural signals and then write for them.

Despite facing various challenges, the team has developed a unique encoding mechanism for Chinese character writing as well as a new technology for decoding the Chinese character writing trajectory.

The team's device can achieve a 91.3-percent accuracy in classifying 100 common Chinese characters offline. With the assistance of a language model, it can achieve online classification accuracy of 96.2 percent.

"The road ahead for brain-computer interface is still long. When the hardware becomes smaller, less invasive, and 'wireless,' we can implant it into some paralyzed patients or patients with ALS to help them perform previously impossible functions, such as controlling an exoskeleton for walking or a robotic arm for daily assistance," said Zhu Junming, deputy head of the neurosurgery department of SAHZU.