Word pair classification during imagined speech using direct brain recordings

Abstract:

People that cannot communicate due to neurological disorders would bene t from an internal speech decoder. Here, we showed the ability to classify individual words during imagined speech from electrocorticographic signals. In a word imagery task, we used high gamma (70–150 Hz) time features with a support vector machine model to classify individual words from a pair of words. To account for temporal irregularities during speech production, we introduced a non-linear time alignment into the SVM kernel. Classi cation accuracy reached 88% in a two-class classi cation framework (50% chance level), and average classi cation accuracy across fteen word-pairs was signi cant across ve subjects (mean = 58%; p < 0.05). We also compared classi cation accuracy between imagined speech, overt speech and listening. As predicted, higher classi cation accuracy was obtained in the listening and overt speech conditions (mean = 89% and 86%, respectively; p < 0.0001), where speech stimuli were directly presented. The results provide evidence for a neural representation for imagined words in the temporal lobe, frontal lobe and sensorimotor cortex, consistent with previous ndings in speech perception and production. These data represent a proof of concept study for basic decoding of speech imagery, and delineate a number of key challenges to usage of speech imagery neural representations for clinical applications.