Communication method and system using brain waves for multidimensional control

• US 5,638,826 A
• Filed: 06/01/1995
• Issued: 06/17/1997
• Est. Priority Date: 06/01/1995
• Status: Expired due to Term

• Current Assignee: Health Research Incorporated
• Original Assignee: Health Research Incorporated
• Inventors: McFarland, Dennis J., Wolpaw, Jonathan R.
• Primary Examiner(s): Cohen, Lee S.
• Assistant Examiner(s): WINAKUR, ERIC FRANK
• Application Number:
  US08/456,758
• Time in Patent Office: 747 Days
• Field of Search: 128/731, 128/732, 273/438, 273/148 B, 340/825.19, 345/157, 341/20, 341/21
• US Class Current: 600/544
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection

Abstract

The invention contemplates that brain waves, preferably mu rhythm waves of an individual, can control a cursor on a video monitor or other device in the environment. Such communication via mu rhythms is optimized both in real time, and by stored data. The mu rhythms are detected, filtered, and amplified, as an individual sits facing a video, or other display screen. The scalp of the individual has electrodes placed in approximately 64 locations, for monitoring the mu rhythms of the alpha band and/or other EEG rhythms. An ear lobe electrode provides a reference. The electrode signals are fed to a channel differential amplifier which amplifies the difference between the 64 channels and the reference electrode. The signals are digitized and spatially filtered. The resulting signals are submitted to a frequency analysis periodically. The powers n specific frequency bands of specific signals are the independent variables for equations used to control cursor movement. Each variable consists of the amplitude at a specific frequency in a specific signal. Two control channels are provided: one for horizontal movement, and the other for vertical movement. Each control channel is defined by a linear or other equation. The intercept of each equation is based on an average of values of the independent variables that have been stored or recorded from the previous performance of the individual. The cursor moves a plurality of times per second in accordance with the analysis scanning frequency. Movement of the cursor to different portions of the screen can control other processes and devices by invoking programmed commands displayed thereon.

Claims

1. A method of exercising control of an object shown upon a display by brain wave monitoring, wherein a brain of an individual being monitored responds to command selections that are shown upon said display, and thereby moves the object shown upon the display to a selection desired by the individual, said method comprising the steps of:

a) monitoring said brain waves of an individual; and

b) computer processing the monitored brain waves in accordance with step (a) in order to control said object shown upon said display, and further including the steps of;

i) monitoring said brain waves at a comprehensive set of head locations;

ii) selecting specific head locations to be utilized to control said object;

iii) feeding said brain waves from said selected head locations to a computer which utilizes temporal and spatial filtering methods to obtain a plurality of object control signals;

vi) using an algorithm to combine said plurality of object control signals to control movement of said object upon said display;

v) storing data consisting of brain waves from all of said monitored head locations, said control signals derived from the selected head locations, and movements of said object; and

vi) periodically analyzing said stored data of step (v), using results of said analysis to modify said selection of specific head locations, said plurality of object control signals derived in step (iii) from said selected head locations, and said algorithm of step (iv), so as to improve control of object movement.

2. The method in accordance with claim 1, further comprising the step of:

c) utilizing said movement of said object shown upon said display to communicate with, or otherwise control a device disposed externally to said computer.

3. The method in accordance with claim 1, wherein the algorithm step (b) (vi) controls the bidirectional movement of said object in at least two dimensions.

4. A method of using brain waves for real time, bidirectional control of a cursor upon a display, in conjunction with stored information, said bidirectional control of said cursor being exhibited along at least two dimensions of said display, and said stored information being used to modify the real time, bidirectional control of said cursor, said method comprising the steps of:

a) selecting head locations on an individual, and detecting and monitoring brain waves of said individual from said select head locations;

b) processing said detected and monitored brain waves from said select head locations of step (a), to obtain object control signals, and using said object control signals in an algorithm, to bidirectionally control movement of a cursor shown upon a display, along at least two dimensions thereof;

c) storing information with respect to the monitored brain waves, derived object control signals, and said algorithm for control of movement of said cursor upon said display in accordance with step (b); and

d) using the information stored in step (c), to modify said selected head locations from step (b), and thereby improve bidirectional control of said cursor movement upon said display in real time.

5. A system of communicating with, or otherwise controlling, a device by using brain waves of an individual, comprising:

monitoring means for detecting and monitoring brain waves of an individual in real time at select head locations;

command selection and feedback means positioned adjacent said individual and operatively connected to said monitoring means, for displaying movement of an object;

an online unit connected to said command selection and feedback means and said monitoring means, for processing and controlling movement of said object displayed by said command selection and feedback means, in response to said real time monitoring of brain waves of said individual by said monitoring means; and

an offline unit connected to said monitoring means for storing and analyzing information with respect to control of said object, said offline unit using said stored information for analysis that results in modifying brain wave control of said object in real time, to thereby optimize real time control of movement of said object by said online unit.

6. The system of communication in accordance with claim 5, further comprising analog-to-digital conversion means connected to said monitoring means and said respective online and offline units, for transforming the detected and monitored brain waves into digital signals.

7. A method of monitoring brain waves of an individual, and displaying movement of an object upon a display in response thereto, and permitting said individual to consciously control said movement with a computer processing the monitored brain waves in order to control said object, including the steps of:

a) choosing select head locations to monitor brain waves;

b) monitoring said select head locations to obtain brain waves;

c) feeding said brain waves to a computer, which uses temporal and spatial filtering routines to eliminate non-EEG artifacts and isolate a plurality of object control signals;

d) combining said plurality of object control signals of step (c) using an algorithm to control movement of said object upon a display in several dimensions; and

e) providing a database consisting of said brain waves, said object control signals and said algorithm to improve control of movement of said object.

8. The method in accordance with claim 7, further comprising the step of:

f) modifying the combining step (d) substantially periodically in real time, in order to enhance control of said object, said control being enhanced by analysis of said stored database of step (e).

9. The method in accordance with claim 7, further comprising the step of:

f) modifying the combining step (d) in real time, in order to enhance control of said object, said control being enhanced through analysis of said stored database of step (e).

10. The method in accordance with claim 7, further comprising the step of:

f) modifying said selected head location choices in accordance with step (a) in real time, in order to enhance control of said object, said control being enhanced through analysis of said stored database of step (e).

11. The method in accordance with claim 7, further comprising the step of:

f) modifying the monitored head locations in accordance with step (b) in real time, in order to enhance control of said object, said control being enhanced through analysis of said stored database of step (e).

12. A system of communicating with, or otherwise controlling, a device by using brain waves of an individual, comprising:

monitoring means for detecting and monitoring said brain waves of an individual in real time at select head locations;

command selection and feedback means positioned adjacent said individual and operatively connected to said monitoring means, for displaying movement of an object;

a first control channel connected to said command selection and feedback means and said monitoring means, for processing and controlling movement of an object in a horizontal plane upon said command selection and feedback means, in response to real time monitoring of brain waves of said individual by said monitoring means; and

a second control channel connected to said command selection and feedback means and said monitoring means, for processing and controlling movement of an object in a vertical plane upon said command selection and feedback means, in response to real time monitoring of brain waves of said individual by said monitoring means.

13. The system in accordance with claim 12, further comprising analog-to-digital conversion means connected to said monitoring means and said respective first and second channels, for transforming the detected and monitored brain waves into digital signals.

14. The system in accordance with claim 13, wherein said first and second control channels each further comprises digital processing means connected to said respective analog-to-digital conversion means for processing the digital signals.

15. A method of processing brain wave signals to control two-dimensional movement of a cursor upon a screen, comprising:

a) monitoring brain waves of an individual to obtain object control signals from select head locations of said individual;

b) directing said object control signals of the monitored brain waves of step (a) into a plurality of processing channels;

c) processing said object control signals in a first one of said plurality of processing channels to control movement of a cursor upon a screen in a first dimension; and

d) simultaneously processing said object control signals in a second one of said plurality of processing channels to control movement of said cursor upon said screen in a second dimension.

16. The method in accordance with claim 15, wherein said processing step (c) includes the steps of:

e) selecting mu rhythm object control signals derived from mu rhythm components of brain waves; and

f) processing said mu rhythm object control signals.

17. The method in accordance with claim 15, wherein said processing step (c) includes the steps of:

e) providing a database of stored brain wave information and object control signals, combined by an algorithm; and

f) using said database of combined brain wave information and said object control signals, to improve object control.

18. The method in accordance with claim 15, wherein said processing step (d) includes the steps of:

selecting at least components of mu rhythms; and

f) processing the mu rhythm components.

19. The method in accordance with claim 15, wherein said processing step (d) includes the steps of:

e) providing a database of stored brain wave information; and

f) processing component brain wave information of said database to improve object control signals.

20. The method in accordance with claim 15, wherein said dimensions of cursor movement in respective steps (c) and (d) are horizontal and vertical, respectively.

21. A method of exercising control of an object shown upon a display by brain wave monitoring, wherein a brain of an individual being monitored responds to command selections that are shown upon said display, thereby moving the object shown upon the display to a selection desired by the individual, said method comprising the steps of:

a) monitoring said brain waves of an individual; and

b) computer-processing the monitored brain waves in accordance with step (a), in order to control said object upon said display, and further including the steps of;

i) monitoring said brain waves at select head locations;

ii) selecting specific head locations to be utilized to control said object;

iii) feeding signals of said brain waves from selected head locations to a computer which utilizes temporal and spatial filtering routines to obtain a plurality of object control signals;

iv) using an algorithm to combine said plurality of object control signals to control movement of said object upon said display;

v) storing data consisting of said brain wave information from all if said monitored, selected head locations;

said control signals derived from said selected head locations; and

information of object movement; and

vi) periodically analyzing said stored data of step (v), using results of said analysis to modify said algorithm of step (iv), so as to improve control of object movement.