17 December 2023 , 14:54
944
0
People's thoughts can be turned into text in a world-first using artificial intelligence.
The mind-reading portable technology could aid communication for people who are unable to speak due to illness or injury, including stroke or paralysis. There is also the potential for the AI system to enable seamless communication between humans and machines, such as for the operation of a robot or a bionic limb.
The tech 'reads' brainwaves via a sensory cap and doesn't involve implants to the brain or any surgery. To test the technology, researchers first had participants silently read passages of text while wearing a cap that recorded electrical brain activity through their scalp using an electroencephalogram (EEG).
They then used their device an AI model called DeWave to segment each person's EEG wave into distinct units that capture specific characteristics and patterns from the human brain. By learning from these large quantities of EEG data, DeWave was ultimately able to translate EEG signals into words and sentences.
Created by scientists from the University of Technology Sydney's GrapheneX-UTS Human-Centric AI Centre, this new device is a step forward from previous brain signal to language technologies such as Elon Musk's Neuralink or a hospital's MRI machine.
Baby boy has spent his life in hospital as doctors are 'scared' to discharge him
The former of these requires surgery to implant electrodes into the brain, making it more invasive and therefore more dangerous, while the latter is large, expensive, and difficult to use in daily life. Both methods also struggle to transform brain signals into individual word segments without the use of additional aids such as eye-tracking, thereby restricting their practical application.
Distinguised Professor Chin-Teng Lin, the study's director, said: "This research represents a pioneering effort in translating raw EEG waves directly into language, marking a significant breakthrough in the field.
"It is the first to incorporate discrete encoding techniques in the brain-to-text translation process, introducing an innovative approach to neural decoding, and the integration with large language models is opening new frontiers in neuroscience and AI."
Professor Lin also highlighted the fact that the study was carried out with 29 participants, likely making it more robust and adaptable than previous decoding technology that has only been tested on one or two individuals. This is because EEG waves can differ significantly between people.
Currently, the system's translation accuracy score is around 40 per cent on BLEU-1 a system which measures the similarity of the machine-translated text to a set of high-quality reference translations. Researchers hope to see this improve over time to a level that is comparable to traditional language translation or speech recognition programmes, which are scored closer to 90 per cent.
Lead author Dr Yigun Duan said: "The model is more adept at matching verbs than nouns."- However, when it comes to nouns, we saw a tendency towards synonymous pairs rather than precise translations, such as 'the man' instead of 'the author'.
"We think this is because when the brain processes these words, semantically similar words might produce similar brain wave patterns. Despite the challenges, our model yields meaningful results, aligning keywords and forming similar sentence structures."
The new study has been selected as the spotlight paper at the NeurIPS conference, a top-tier annual meeting that showcases world-leading research on AI and machine learning. It follows on from previous brain-computer interface technology developed by the University of Technology Sydney, which, in collaboration with the Australian Defence Force, used brainwaves to command a quadruped robot.