Cognitive control signals for neural prosthetics

• US 7,826,894 B2
• Filed: 03/22/2005
• Issued: 11/02/2010
• Est. Priority Date: 03/22/2004
• Status: Active Grant

• Current Assignee: California Institute of Technology
• Original Assignee: California Institute of Technology
• Inventors: Musallam, Sam, Greger, Bradley, Scherberger, Hansjorg, Corneil, Brian D., Andersen, Richard A.
• Primary Examiner(s): Hindenburg; Max
• Assistant Examiner(s): TOWA, RENE T
• Application Number:
  US11/086,534
• Publication Number:
  US 20050228515A1
• Time in Patent Office: 2,051 Days
• Field of Search: 600/300, 600544-546, 600/378, 706/12, 706/47, 623 24- 25, 623/27, 623/57
• US Class Current: 600/544
• CPC Class Codes: A61F 2002/704 computer-controlled, e.g. robotic control
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06N 3/061 using biological neurons, e.g. biological neurons connected to an integrated circuit

Abstract

In an embodiment, the invention relates to neural prosthetic devices in which control signals are based on the cognitive activity of the prosthetic user. The control signals may be used to control an array of external devices, such as prosthetics, computer systems, and speech synthesizers. Data obtained from monkeys'"'"' movement intentions were recorded, decoded with a computer algorithm, and used to position cursors on a computer screen. Not only the intended goals, but also the value of the reward the animals expected to receive at the end of each trial, were decoded from the recordings. The results indicate that brain activity related to cognitive variables can be a viable source of signals for the control of a cognitive-based neural prosthetic.

Claims

1. A method, comprising:

implanting at least one electrode in a brain of a subject;

receiving electrical signals generated by a plurality of neurons at the at least one implanted electrode, the electrical signals comprising a portion that encodes at least one abstract cognitive concept, the portion of the electrical signals encoding the at least one abstract cognitive concept not encoding either motor movements or intended motor movements, the plurality of neurons having generated the portion of the electrical signals encoding the at least one abstract cognitive concept without having been trained to do so;

at least partially decoding the at least one abstract cognitive concept encoded in the portion of the electrical signals by generating a feature set based on the electrical signals and mapping the feature set to the at least one abstract cognitive concept using a transformation rule; and

generating a control signal to instruct an external device to perform a task associated with the at least partially decoded at least one abstract cognitive concept.

2. The method of claim 1, wherein the at least one abstract cognitive concept encoded in the portion of the electrical signals is an intended goal of the subject.

3. The method of claim 1, wherein the at least one abstract cognitive concept encoded in the portion of the electrical signals is an expected value, speech segment, abstract thought, executive control, attention, decision, motivation, memory, imagery, emotion, or mood of the subject.

4. The method of claim 1, further comprising building a database of neural activity data detected from the subject.

5. The method of claim 4, wherein the database comprises information collected from at least one brain control trial performed by the subject.

6. The method of claim 1, wherein at least a portion of the plurality of neurons are located in at least one area of the brain of the subject that generates motor signals.

7. The method of claim 1, further comprising causing the external device to perform the task.

8. The method of claim 1, wherein the external device is a prosthetic limb.

9. The method of claim 1, wherein the external device is selected from a group consisting of a computer device, a speech synthesizer, a wheelchair, a vehicle, a controller, a touch pad, a keyboard, a robotic hand, and combinations thereof.

10. The method of claim 1, wherein generating a control signal further comprises deriving multiple control signals from a single area in the brain or from a simultaneous recording from multiple brain areas.

11. The method of claim 1, wherein generating a control signal further comprises deriving multiple control signals from local field potential (“

LFP”

) recordings, spike activity, or combinations thereof.

12. The method of claim 1, wherein the control signal is an expected value signal.

13. The method of claim 1, wherein the task associated with the at least partially decoded at least one abstract cognitive concept comprises determining the nature of the at least one abstract cognitive concept itself, wherein the at least one abstract cognitive concept is selected from the group consisting of a mood, a motivation, an attention, a preference, a decision, and combinations thereof.

14. The method of claim 1, further comprising improving the subject'"'"'s control of the external device based on direct effort and learning.

15. The method of claim 1, wherein the at least one abstract cognitive concept encoded by the portion of the electrical signals comprises a preference for a first option over at least one other option, and at least partially decoding the at least one abstract cognitive concept comprises identifying the preference for the first option over the at least one other option.

16. The method of claim 15, wherein the task performed by the external device comprises selecting the first option.

17. The method of claim 1, further comprising:

deriving the transformation rule by (a) determining a probabilistic relationship between the feature set and the at least one abstract cognitive concept, and (b) minimizing an expected loss which represents a risk associated with incorrectly decoding the at least one abstract cognitive concept to be other than the at least one abstract cognitive concept encoded by the portion of the electrical signals.

18. The method of claim 1, further comprising:

deriving the transformation rule by (a) constructing a neural network using a training set comprising plural feature sets each associated to an abstract cognitive concept and (b) minimizing an aggregate expected loss for the training set which represents an aggregate risk associated with incorrectly decoding the abstract cognitive concepts from the plural feature sets.

19. A non-transitory machine-readable medium storing machine-executable instructions that when executed by a machine cause the machine to:

detect neural activity within electrical signals generated by a plurality of neurons that encodes at least one abstract cognitive concept, the detected neural activity being distinguishable from neural activity that encodes information related to both motor movements and intended motor movements, the electrical signals having been received from at least one electrode implanted in at least one area of a brain of a subject, the subject being untrained with respect to generating the detected neural activity;

generate a feature set for the detected neural activity;

map the feature set to the at least one abstract cognitive concept using a transformation rule; and

generate a control signal to instruct an external device to perform a task associated with the at least one abstract cognitive concept encoded by the detected neural activity.

20. The non-transitory machine-readable medium of claim 19, wherein the at least one abstract cognitive concept encoded in the detected neural activity is an intended goal of the subject.

21. The non-transitory machine-readable medium of claim 19, wherein the at least one abstract cognitive concept encoded in the detected neural activity is an expected value, speech segment, abstract thought, executive control, attention, decision, motivation, memory, imagery, emotion or mood of the subject.

22. The non-transitory machine-readable medium of claim 19, wherein the instructions further cause the machine to build a database of detected neural activity.

23. The non-transitory machine-readable medium of claim 22, wherein the database comprises information collected from at least one brain control trial performed by the subject.

24. The non-transitory machine-readable medium of claim 19, wherein the instructions further cause the machine to cause the external device to perform the task.

25. The non-transitory machine-readable medium of claim 19, wherein the external device is a prosthetic limb.

26. The non-transitory machine-readable medium of claim 19, wherein the external device is selected from the group consisting of a computer device, a speech synthesizer, a wheelchair, a vehicle, a controller, a touch pad, a keyboard, a robotic hand, and combinations thereof.

27. The non-transitory machine-readable medium of claim 19, wherein the instructions to generate the control signal further comprise instructions to derive multiple control signals from a single area in the brain or from a simultaneous recording from multiple brain areas.

28. The non-transitory machine-readable medium of claim 19, wherein the instructions to generate the control signal further comprise instructions to derive multiple control signals from local field potential (“

LFP”

) recordings, spike activity, or combinations thereof.

29. The non-transitory machine-readable medium of claim 19, wherein the control signal is an expected value signal.

30. The non-transitory machine-readable medium of claim 19, wherein the task associated with the at least one abstract cognitive concept encoded by the detected neural activity comprises determining the nature of the at least one abstract cognitive concept itself, wherein the at least one abstract cognitive concept is selected from the group consisting of a mood, a motivation, an attention, a preference, a decision, and combinations thereof.

31. An apparatus, comprising:

a measuring device implantable in a subject;

a database of neural activity data observed in the subject;

an input device operative to receive electrical signals from the measuring device after the measuring device is implanted in the subject, the electrical signals comprising neural activity generated by a plurality of neurons and encoding at least one abstract cognitive concept, the neural activity encoding the at least one abstract cognitive concept being different from neural activity encoding information related to both motor movements and intended motor movements, the plurality of neurons having not been trained to generate the neural activity encoding the at least one abstract cognitive concept;

a signal processor connected to the input device, and operative to analyze the neural activity encoding the at least one abstract cognitive concept to generate a feature set; and

a decoder operative to apply a transformation rule to the feature set that maps the feature set to the at least one abstract cognitive concept encoded by the neural activity, and generate a control signal based on an analytical operation performed with the database of neural activity data from the subject.

32. The apparatus of claim 31, wherein the at least one abstract cognitive concept encoded by the neural activity is an intended goal of the subject.

33. The apparatus of claim 31, wherein the at least one abstract cognitive concept encoded by the neural activity is an expected value, speech segment, abstract thought, executive control, attention, decision, motivation, memory, imagery, emotion, or mood of the subject.

34. The apparatus of claim 31, wherein the database comprises information collected from at least one brain control trial performed by the subject.