Computational models that mimic the structure and performance of the human auditory system could help researchers design higher hearing aids, cochlear implants, and brain-machine interfaces. A brand new study from MIT has found that modern computational models derived from machine learning are moving closer to this goal.
In the biggest study yet of deep neural networks which have been trained to perform auditory tasks, the MIT team showed that almost all of those models generate internal representations that share properties of representations seen within the human brain when individuals are listening to the identical sounds.
The study also offers insight into the right way to best train the sort of model: The researchers found that models trained on auditory input including background noise more closely mimic the activation patterns of the human auditory cortex.
“What sets this study apart is it’s essentially the most comprehensive comparison of those sorts of models to the auditory system up to now. The study suggests that models which can be derived from machine learning are a step in the precise direction, and it gives us some clues as to what tends to make them higher models of the brain,” says Josh McDermott, an associate professor of brain and cognitive sciences at MIT, a member of MIT’s McGovern Institute for Brain Research and Center for Brains, Minds, and Machines, and the senior creator of the study.
MIT graduate student Greta Tuckute and Jenelle Feather PhD ’22 are the lead authors of the open-access paper, which appears today in PLOS Biology.
Models of hearing
Deep neural networks are computational models that consists of many layers of information-processing units that could be trained on huge volumes of knowledge to perform specific tasks. Any such model has change into widely utilized in many applications, and neuroscientists have begun to explore the chance that these systems will also be used to explain how the human brain performs certain tasks.
“These models which can be built with machine learning are capable of mediate behaviors on a scale that actually wasn’t possible with previous forms of models, and that has led to interest in whether or not the representations within the models might capture things which can be happening within the brain,” Tuckute says.
When a neural network is performing a task, its processing units generate activation patterns in response to every audio input it receives, akin to a word or other variety of sound. Those model representations of the input could be in comparison with the activation patterns seen in fMRI brain scans of individuals listening to the identical input.
In 2018, McDermott and then-graduate student Alexander Kell reported that once they trained a neural network to perform auditory tasks (akin to recognizing words from an audio signal), the inner representations generated by the model showed similarity to those seen in fMRI scans of individuals listening to the identical sounds.
Since then, all these models have change into widely used, so McDermott’s research group set out to guage a bigger set of models, to see if the flexibility to approximate the neural representations seen within the human brain is a general trait of those models.
For this study, the researchers analyzed nine publicly available deep neural network models that had been trained to perform auditory tasks, and in addition they created 14 models of their very own, based on two different architectures. Most of those models were trained to perform a single task — recognizing words, identifying the speaker, recognizing environmental sounds, and identifying musical genre — while two of them were trained to perform multiple tasks.
When the researchers presented these models with natural sounds that had been used as stimuli in human fMRI experiments, they found that the inner model representations tended to exhibit similarity with those generated by the human brain. The models whose representations were most much like those seen within the brain were models that had been trained on a couple of task and had been trained on auditory input that included background noise.
“In case you train models in noise, they offer higher brain predictions than in case you don’t, which is intuitively reasonable because a number of real-world hearing involves hearing in noise, and that’s plausibly something the auditory system is tailored to,” Feather says.
Hierarchical processing
The brand new study also supports the concept that the human auditory cortex has some extent of hierarchical organization, wherein processing is split into stages that support distinct computational functions. As within the 2018 study, the researchers found that representations generated in earlier stages of the model most closely resemble those seen in the first auditory cortex, while representations generated in later model stages more closely resemble those generated in brain regions beyond the first cortex.
Moreover, the researchers found that models that had been trained on different tasks were higher at replicating different points of audition. For instance, models trained on a speech-related task more closely resembled speech-selective areas.
“Though the model has seen the very same training data and the architecture is identical, while you optimize for one particular task, you’ll be able to see that it selectively explains specific tuning properties within the brain,” Tuckute says.
McDermott’s lab now plans to utilize their findings to attempt to develop models which can be much more successful at reproducing human brain responses. Along with helping scientists learn more about how the brain could also be organized, such models is also used to assist develop higher hearing aids, cochlear implants, and brain-machine interfaces.
“A goal of our field is to find yourself with a pc model that may predict brain responses and behavior. We predict that if we’re successful in reaching that goal, it can open a number of doors,” McDermott says.
The research was funded by the National Institutes of Health, an Amazon Fellowship from the Science Hub, an International Doctoral Fellowship from the American Association of University Women, an MIT Friends of McGovern Institute Fellowship, a fellowship from the K. Lisa Yang Integrative Computational Neuroscience (ICoN) Center at MIT, and a Department of Energy Computational Science Graduate Fellowship.
