Methods of Decoding Speech from the Brain and Systems for Practicing the Same

• US 20150380009A1
• Filed: 02/19/2014
• Published: 12/31/2015
• Est. Priority Date: 02/19/2013
• Status: Active Grant

• Current Assignee: Regents of the University of California (University of California)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: CHANG, Edward F., BOUCHARD, Kristofer E.
• Granted Patent: US 9,905,239 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 5/24 Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
• CPC Class Codes: A61B 5/4076 Diagnosing or monitoring particular conditions of the nervous system A61B5/4821, A61B5/4824 take precedence
• CPC Class Codes: A61B 5/4094 Diagnosing or monitoring seizure diseases, e.g. epilepsy
• CPC Class Codes: A61B 5/4803 Speech analysis specially adapted for diagnostic purposes
• CPC Class Codes: A61B 5/7203 for noise prevention, reduction or removal
• CPC Class Codes: A61B 5/7257 using Fourier transforms
• CPC Class Codes: A61B 5/726 using Wavelet transforms
• CPC Class Codes: G09B 21/00 Teaching, or communicating with, the blind, deaf or mute audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: G10L 25/03 characterised by the type of extracted parameters
• CPC Class Codes: G10L 25/06 the extracted parameters being correlation coefficients

Abstract

Provided are methods of decoding speech from the brain of a subject. The methods include detecting speech production signals from electrodes operably coupled to the speech motor cortex of a subject while the subject produces or imagines producing a speech sound. The methods further include deriving a speech production signal pattern from the detected speech production signals, and correlating the speech production signal pattern with a reference speech production signal pattern to decode speech from the brain of the subject. Speech communication systems and devices for practicing the subject methods are also provided.

Claims

1. A method of decoding speech from the brain of a subject, comprising:

detecting speech production signals from at least three electrodes operably coupled to the speech motor cortex of a subject while the subject produces or imagines producing a speech sound;

deriving a speech production signal pattern from the detected speech production signals; and

correlating the speech production signal pattern with a reference speech production signal pattern to decode speech from the brain of the subject.

2. The method of claim 1, wherein the at least three electrodes comprises at least 10 electrodes.

3. The method of claim 1, wherein the at least three electrodes comprises at least 20 electrodes.

4. The method of claim 1, wherein the speech production signals are detected from the ventral sensorimotor cortex.

5. The method of claim 4, wherein the speech production signals are detected from a region of the ventral sensorimotor cortex selected from:

the pre-central gyrus, the post-central gyrus, and combinations thereof.

6. The method of claim 1, wherein the at least three electrodes are operably coupled to the speech motor cortex by implantation on the surface of the speech motor cortex.

7. The method of claim 6, wherein the speech production signals are detected using electrocorticography (ECoG).

8. The method of claim 1, wherein the at least three electrodes are operably coupled to the speech motor cortex by insertion of the electrodes into the speech motor cortex.

9. The method of claim 1, wherein detecting speech production signals comprises detecting local field potentials from the speech motor cortex using the at least three electrodes.

10. The method of claim 9, wherein the speech production signals comprise the high-gamma frequency component (85-175 Hz) of the local field potentials.

11. The method of claim 1, wherein the detecting comprises detecting a time-course of speech production signals from each of the least three electrodes.

12. The method of claim 11, wherein the time-course of speech production signals is detected beginning from 250 milliseconds or more prior to the onset of the speech sound.

13. The method of claim 1, wherein the speech sound is selected from the group consisting of:

a phoneme, formant acoustics of a vowel, a diphone, a triphone, a consonant-vowel transition, a syllable, a word, a phrase, a sentence, and combinations thereof.

14. The method of claim 13, wherein the speech sound is a syllable.

15. The method of claim 1, wherein deriving a speech production signal pattern comprises performing time-frequency analysis of speech production signals detected from each of the at least three electrodes.

16. The method of claim 15, wherein the time-frequency analysis is performed using a method selected from the group consisting of:

Fast Fourier Transform (FFT), wavelet transform, Hilbert transform, and bandpass filtering.

17. The method of claim 1, wherein the at least three electrodes are distributed across the speech motor cortex such that speech production signals corresponding to at least three speech articulators are detected.

18. The method of claim 17, wherein the at least three speech articulators are selected from the group consisting of:

the larynx, the lips, the jaw, and the tongue.

19. The method of claim 1, wherein the subject has a speech impairment.

20-41. -41. (canceled)