Wearing an electrode-studded cap bristling with wires, a young man silently reads a sentence in his head. Moments later, a Siri-like voice breaks in, attempting to translate his thoughts into text, “Yes, I’d like a bowl of chicken soup, please.” It’s the newest example of computers translating an individual’s thoughts into words and sentences.
Previously, researchers have used implants surgically placed within the brain or bulky, expensive machines to translate brain activity into text. The recent approach, presented at this week’s NeurIPS conference by researchers from the University of Technology Sydney, is impressive for its use of a non-invasive EEG cap and the potential to generalize beyond one or two people.
The team built an AI model called DeWave that’s trained on brain activity and language and linked it as much as a big language model—the technology behind ChatGPT—to assist convert brain activity into words. In a preprint posted on arXiv, the model beat previous top marks for EEG thought-to-text translation with an accuracy of roughly 40 percent. Chin-Teng Lin, corresponding creator on the paper, told MSN they’ve more recently upped the accuracy to 60 percent. The outcomes are still being peer-reviewed.
Though there’s a protracted option to go by way of reliability, it shows progress in non-invasive methods of reading and translating thoughts into language. The team believes their work could give voice to those that can now not communicate as a result of injury or disease or be used to direct machines, like walking robots or robotic arms, with thoughts alone.
Guess What I’m Considering
It’s possible you’ll remember headlines about “mind-reading” machines translating thoughts to text at high speed. That’s because such efforts are hardly recent.
Earlier this 12 months, Stanford researchers described work with a patient, Pat Bennett, who’d lost the power to talk as a result of ALS. After implanting 4 sensors into two parts of her brain and extensive training, Bennett could communicate by having her thoughts converted to text at a speed of 62 words per minute—an improvement on the identical team’s 2021 record of 18 words per minute.
It’s a tremendous result, but brain implants could be dangerous. Scientists would like to get an analogous end result without surgery.
In one other study this 12 months, researchers on the University of Texas at Austin turned to a brain-scanning technology called fMRI. Within the study, patients needed to lie very still in a machine recording the blood flow of their brains as they listened to stories. After using this data to a train an algorithm—based partly on ChatGPT ancestor, GPT-1—the team used the system to guess what participants were hearing based on their brain activity.
The system’s accuracy wasn’t perfect, it required heavy customization for every participant, and fMRI machines are bulky and expensive. Still, the study served as a proof of concept that thoughts could be decoded non-invasively, and the newest in AI may help make it occur.
The Sorting Hat
In Harry Potter, students are sorted into school houses by a magical hat that reads minds. We muggles resort to funny looking swim caps punctured by wires and electrodes. Referred to as electroencephalograph (EEG) caps, these devices read and record the electrical activity in our brains. In contrast with brain implants, they require no surgery but are considerably less accurate. The challenge, then, is to separate signal from noise to get a useful result.
In the brand new study, the team used two datasets containing eye-tracking and EEG recordings from 12 and 18 people, respectively, as they read text. Eye-tracking data helped the system slice up brain activity by word. That’s, when an individual’s eyes flit from one word to the following, it means there must be a break between the brain activity related to that word and the activity that must be correlated with the following one.
They then trained DeWave on this data, and over time, the algorithm learned to associate particular brain wave patterns with words. Finally, with the assistance of a pre-trained large language model called BART—fine-tuned to grasp the model’s unique output—the algorithm’s brain-wave-to-word associations were translated back into sentences.
In tests, DeWave outperformed top algorithms within the category in each the interpretation of raw brain waves and brain waves sliced up by word. The latter were more accurate, but still lagged way behind translation between languages—like English and French—and speech recognition. Additionally they found the algorithm performed similarly across participants. Prior experiments have tended to report results for one person or require extreme customization.
The team says the research is more proof large language models may help advance brain-to-text systems. Although they used a comparatively antique algorithm within the official study, in supplementary material they included results from larger models, including Meta’s original Llama algorithm. Interestingly, the larger algorithms didn’t improve results much.
“This underscores the complexity of the issue and the challenges of bridging brain activities with LLMs,” the authors wrote, calling for more nuanced research in the longer term. Still, the team hopes they’ll push their very own system further, perhaps as much as 90 percent accuracy.
The work shows progress in the sphere.
“People have been wanting to show EEG into text for a very long time and the team’s model is showing a remarkable amount of correctness,” the University of Sydney’s Craig Jin told MSN. “Several years ago the conversions from EEG to text were complete and utter nonsense.”
