Sound meets meaning: An orchestration of coupled non-linear systems to incorporate transient dynamics with oscillatory inter-areal cortical communication.

Objective

ORCHESTRAS posits a new a theoretical framework for a mechanistic understanding of neural tracking of speech by synthesizing views from both the oscillatory and evoked-like paradigms. It builds upon work from computational neuroscience on whole-brain modelling together with the emergent field of NeuroAI where artificial neural networks are augmented with neural architectural priors which serves in bringing inductive biases to AI models that allow for faster learning and more efficient inference. ORCHESTRAS’ oscillator-based model naturally aligns with the concept of neural oscillations as a mechanism for prediction and expectation. The core hypothesis lies in the idea that event-based predictions (e.g. after hearing a context leading to a strong word or phoneme prediction) elicit a transient response which synchronises slow-phase signal between areas. In turn, that phase modulates higher frequency oscillations reflected in power increase observed in sentence building or upon unexpected words. The interplay between transient and oscillatory dynamic via phase-amplitude coupling is the key linkage of the proposed modelling approach. In other words, we are looking at integrating evoked potentials dynamically with oscillatory activity, a synergic interaction allowing to map linguistic predictions to oscillatory brain activity. The project will be developed along two main approaches: (1) consolidating our theoretical understanding of neural tracking of speech using whole-brain modelling of non-linear oscillators and (2) bridging with AI systems for language modelling such as Transformers by enhancing them with novel attention mechanism based on phase-ampltidude coupling akin to the biological brain. The findings are expected to have significant impact in both our theoretical understanding of speech processing in the human brain and in our practical usage of large language models in AI systems.