Direct classification of all American English phonemes using signals from functional speech motor cortex

Emily M. Mugler, James L. Patton, Robert D. Flint, Zachary A. Wright, Stephan U. Schuele, Joshua Rosenow, Jerry J. Shih, Dean J. Krusienski, Marc W. Slutzky

Research output: Contribution to journalArticle

77 Scopus citations

Abstract

Objective. Although brain-computer interfaces (BCIs) can be used in several different ways to restore communication, communicative BCI has not approached the rate or efficiency of natural human speech. Electrocorticography (ECoG) has precise spatiotemporal resolution that enables recording of brain activity distributed over a wide area of cortex, such as during speech production. In this study, we sought to decode elements of speech production using ECoG. Approach. We investigated words that contain the entire set of phonemes in the general American accent using ECoG with four subjects. Using a linear classifier, we evaluated the degree to which individual phonemes within each word could be correctly identified from cortical signal. Main results. We classified phonemes with up to 36% accuracy when classifying all phonemes and up to 63% accuracy for a single phoneme. Further, misclassified phonemes follow articulation organization described in phonology literature, aiding classification of whole words. Precise temporal alignment to phoneme onset was crucial for classification success. Significance. We identified specific spatiotemporal features that aid classification, which could guide future applications. Word identification was equivalent to information transfer rates as high as 3.0 bits s-1 (33.6 words min-1), supporting pursuit of speech articulation for BCI control.

Original languageEnglish (US)
Article number035015
JournalJournal of neural engineering
Volume11
Issue number3
DOIs
StatePublished - Jun 2014

Keywords

  • Brain-computer interface
  • Electrocorticography
  • Linear discriminant analysis
  • Phonemes
  • Speech production

ASJC Scopus subject areas

  • Biomedical Engineering
  • Cellular and Molecular Neuroscience

Fingerprint Dive into the research topics of 'Direct classification of all American English phonemes using signals from functional speech motor cortex'. Together they form a unique fingerprint.

  • Cite this

    Mugler, E. M., Patton, J. L., Flint, R. D., Wright, Z. A., Schuele, S. U., Rosenow, J., Shih, J. J., Krusienski, D. J., & Slutzky, M. W. (2014). Direct classification of all American English phonemes using signals from functional speech motor cortex. Journal of neural engineering, 11(3), [035015]. https://doi.org/10.1088/1741-2560/11/3/035015