METHODS OF DECODING SPEECH FROM BRAIN ACTIVITY DATA AND DEVICES FOR PRACTICING THE SAME

• US 20150297106A1
• Filed: 10/25/2013
• Published: 10/22/2015
• Est. Priority Date: 10/26/2012
• Status: Active Grant

• Current Assignee: Regents of the University of California (University of California)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Knight, Robert T., Pasley, Brian
• Granted Patent: US 10,264,990 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 5/0006 ECG or EEG signals
• CPC Class Codes: A61B 5/291 for electroencephalography [EEG]
• CPC Class Codes: A61B 5/316 Modalities, i.e. specific diagnostic methods
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/374 Detecting the frequency distribution of signals, e.g. detecting delta, theta, alpha, beta or gamma waves
• CPC Class Codes: A61B 5/38 Acoustic or auditory stimuli
• CPC Class Codes: A61B 5/4803 Speech analysis specially adapted for diagnostic purposes
• CPC Class Codes: A61B 5/6814 Head
• CPC Class Codes: A61B 5/6868 Brain
• CPC Class Codes: A61B 5/6898 Portable consumer electronic devices, e.g. music players, telephones, tablet computers
• CPC Class Codes: A61B 5/741 using synthesised speech
• CPC Class Codes: A61B 5/742 using visual displays A61B5/339 takes precedence
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

The present disclosure provides methods of decoding speech from brain activity data. Aspects of the methods include receiving brain speech activity data from a subject, and processing the brain speech activity data to output speech feature data. Also provided are devices and systems for practicing the subject methods.

Claims

1. A method of decoding brain speech activity of a subject into speech feature data, the method comprising:

(a) receiving brain speech activity data from the subject into a decoder device;

(b) processing the received brain speech activity data with a decoder module present in the decoder device, wherein the decoder module is configured to convert the received brain speech activity data into speech feature data; and

(c) outputting the speech feature data to an output device.

2. The method according to claim 1, wherein the method further comprises obtaining the brain speech activity data from the subject.

3. The method according to claim 2, wherein the brain speech activity data is obtained from a region of the subject'"'"'s brain associated with speech recognition.

4. The method according to claim 3, wherein the region comprises the peri-sylvian language cortex including the posterior superior temporal gyrus (pSTG), motor cortex, Broca'"'"'s area and combinations thereof.

5. The method according to claim 2, wherein the brain speech activity data is obtained from an electrode device.

6. The method according to claim 5, wherein the electrode device comprises a plurality of electrodes.

7. The method according to claim 6, wherein the electrode device is implanted.

8. The method according to claim 7, wherein the brain speech activity data is electrocorticography (ECoG) data.

9. The method according to claim 6, wherein the electrode device is not implanted.

10. The method according to claim 9, wherein the brain speech activity data is electroencephalography (EEG) data.

11. The method according to claim 1, wherein the decoder module employs at least one of a linear spectrogram model and a non-linear modulation model.

12. The method according to claim 11, wherein the decoder module is configured to select between a linear spectrogram model and a non-linear modulation model.

13. The method according to claim 1, wherein the speech feature data is configured to be converted by an output device into readable text.

14. The method according to claim 1, wherein the speech feature data is configured to be converted by an output device into audible sound.

15. The method according to claim 14, wherein the audible sound is a human recognizable word.

16. The method according to claim 14, wherein the audible sound is a human recognizable collection of words.

17. The method according to claim 16, wherein the human recognizable collection comprises phrases or sentences.

18. The method according to claim 1, wherein the brain speech activity data is the result of heard speech.

19. The method according to claim 1, wherein the brain speech activity data is the result of imagined speech.

20-24. -24. (canceled)

25. A system for decoding brain speech activity of a subject into speech feature data, the system comprising:

a decoder device comprising;

a data input component configured to receive brain speech activity data;

a decoder module comprising;

a processor; and

a machine-readable medium encoding instructions operable to;

cause the processor to convert received brain speech activity data into speech feature data; and

output the speech feature data to an output component; and

a data output component configured to output speech feature data from the decoder module.

26-45. -45. (canceled)