Method for controlling device by using brain wave and brain wave interface system

• US 8,473,045 B2
• Filed: 03/03/2010
• Issued: 06/25/2013
• Est. Priority Date: 07/11/2008
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Terada, Yoshihisa, Morikawa, Koji
• Primary Examiner(s): Natnithithadha, Navin
• Assistant Examiner(s): BERHANU, ETSUB D
• Application Number:
  US12/716,425
• Publication Number:
  US 20100191140A1
• Time in Patent Office: 1,210 Days
• Field of Search: 600/544, 600/545
• US Class Current: 600/545
• CPC Class Codes: A61B 5/378 Visual stimuli
• CPC Class Codes: A61B 5/6814 Head
• CPC Class Codes: A61B 5/7203 for noise prevention, reduction or removal
• CPC Class Codes: A61B 5/7207 of noise induced by motion artifacts
• CPC Class Codes: A61B 5/726 using Wavelet transforms
• CPC Class Codes: A61B 5/7475 User input or interface means, e.g. keyboard, pointing device, joystick
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection

Abstract

The control method for a device includes steps of: presenting a visual stimulation concerning a manipulation menu for a device; measuring event-related potentials after the visual stimulation is presented, where event-related potentials based on a timing of presenting the visual stimulation as a starting point are measured from a potential difference between each of electrodes and at least one reference electrode respectively worn on a face and in an ear periphery of a user; from each of the measured event-related potentials, extracting electroencephalogram data which is at 5 Hz or less and contains a predetermined time section, and combining the extracted electroencephalogram data into electroencephalogram characteristic data; comparing the electroencephalogram characteristic data against reference data prepared in advance for determining a desire to select an item in the manipulation menu; and, based on a comparison result, executing a manipulation of the device corresponding to the item.

Claims

1. A control method for a device utilizing an electroencephalogram, comprising:

step (a) of presenting a visual stimulation concerning a manipulation menu for a device;

step (b) of measuring a plurality of event-related potentials after the visual stimulation is presented, where a plurality of event-related potentials based on a timing of presenting the visual stimulation as a starting point are measured from a potential difference between each of a plurality of electrodes worn on a face of a user and at least one reference electrode worn in an ear periphery of the user;

step (c) of, from each of the plurality of measured event-related potentials, and using an electroencephalogram characteristic extraction section, extracting electroencephalogram data which is at 5 Hz or less and contains a predetermined time section, wherein the predetermined time section is a zone from 200 ms to 400 ms based on the presenting of the visual stimulation as a starting point, and combining the extracted electroencephalogram data into electroencephalogram characteristic data;

step (d) of comparing the electroencephalogram characteristic data against reference data using a determination section, the reference data being prepared in advance for determining a desire to select an item in the manipulation menu; and

step (e) of, based on a result of comparison of step (d), executing a manipulation of the device corresponding to the item in the manipulation menu.

2. The control method of claim 1, wherein step (b) measures the plurality of event-related potentials by using at least one reference electrode worn in each one of two ear peripheries of the user.

3. The control method of claim 2, further comprising:

step (f) of, based on electrical characteristics between each of the plurality of electrodes worn on the face of the user and the at least one reference electrode worn in each one of two ear peripheries of the user, determining a state of attachment of each of the plurality of electrodes and the at least one reference electrode; and

step (g) of, based on a result of determination of step (f), determining a combination of electrodes for measuring the plurality of event-related potentials at step (b).

4. The control method of claim 3, wherein step (g) detects, among combinations of electrodes obtained by respectively combining a plurality of electrodes worn on the face of the user and the at least one reference electrode worn in each one of two ear peripheries of the user, a plurality of combinations of electrodes of which measured values of the plurality of event-related potentials each exceed a threshold value, and searches for an electrode which is commonly included among the plurality of combinations of electrodes to identify an electrode whose state of attachment is insufficient.

5. The control method of claim 4, wherein the electrode identified at step (g) is notified in a distinguishable manner.

6. The control method of claim 2, wherein step (b) measures the plurality of event-related potentials by using an electrode worn in at least one of a position above a right eye and a position above a left eye of the user.

7. The control method of claim 1, wherein step (b) measures the plurality of event-related potentials by using an electrode worn in at least one of a position above a right eye and a position above a left eye of the user.

8. The control method of claim 1, wherein step (c) extracts, from waveforms of the plurality of measured event-related potentials, electroencephalogram data representing characteristic features of the waveforms in terms of time and frequency.

9. The control method of claim 8, wherein step (c) subjects the measured waveforms of the electroencephalograms to a wavelet transform to extract electroencephalogram data representing characteristic features of the waveforms in terms of time and frequency.

10. The control method of claim 1, wherein step (c) extracts, from waveforms of the plurality of event-related potentials measured with the at least one reference electrode in the ear periphery and the plurality of electrodes on the face, data representing a characteristic feature of each waveform, and generates a single piece of electroencephalogram characteristic data based on the respective data.

11. The control method of claim 1, wherein step (b) of measuring a plurality of event-related potentials after the visual stimulation is presented includes measuring a potential difference between an electrode worn alongside the left eye and the at least one reference electrode worn in the right ear periphery.

12. The control method of claim 1, wherein step (b) of measuring a plurality of event-related potentials after the visual stimulation is presented includes measuring a potential difference between an electrode worn alongside the right eye and the at least one reference electrode worn in the left ear periphery.

13. An electroencephalogram interface system comprising:

an output section for visually presenting a manipulation menu;

a plurality of electrodes respectively configured to be worn in an ear periphery and on a face of a user for measuring electroencephalograms of the user;

an electroencephalogram characteristic extraction section configured to extract electroencephalogram data which is at 5 Hz or less and contains a predetermined time section from each of a plurality of event-related potentials measured from potential differences between a plurality of electrodes configured to be worn on the face and at least one reference electrode configured to be worn in the ear periphery based on a timing of presenting the manipulation menu as a starting point, wherein the predetermined time section is a zone from 200 ms to 400 ms based on the presenting of the manipulation menu as a starting point, and combining the extracted electroencephalogram data into electroencephalogram characteristic data representing characteristic features of waveforms of the plurality of measured event-related potentials; and

a determination section configured to determine a similarity level by comparing the electroencephalogram characteristic data against reference data which is prepared in advance for determining a desire to select an item in the manipulation menu, and controlling a device based on a result of determination.

14. The electroencephalogram interface system of claim 13, wherein,

the output section is a display;

and

the determination section controls a displayed substance on the display based on the result of determination.

15. The electroencephalogram interface system of claim 14, further comprising a transmission section for outputting a control signal for an external device,

wherein the determination section outputs the control signal based on the result of determination, and controls an operation of the external device based on the control signal.

16. A control device for a display device, the control device and the display device together constituting an electroencephalogram interface system, the control device comprising:

a communication section for communicating with the display device to cause the display device to visually present a manipulation menu;

a plurality of electrodes respectively configured to be worn in an ear periphery and on a face of a user for measuring electroencephalograms of the user;

an electroencephalogram characteristic extraction section configured to extract electroencephalogram data which is at 5 Hz or less and contains a predetermined time section from each of a plurality of event-related potentials measured from potential differences between a plurality of electrodes configured to be worn on the face and at least one reference electrode configured to be worn in the ear periphery based on a timing of presenting the manipulation menu as a starting point, wherein the predetermined time section is a zone from 200 ms to 400 ms based on the presenting of the manipulation menu as a starting point, and combining the extracted electroencephalogram data into electroencephalogram characteristic data representing characteristic features of waveforms of the plurality of measured event-related potentials; and

a determination section configured to determine a similarity level by comparing the electroencephalogram characteristic data against reference data which is prepared in advance for determining a desire to select an item in the manipulation menu, and controlling a device based on a result of determination.