Method and system for detecting and classifying facial muscle movements

• US 20070060830A1
• Filed: 09/12/2005
• Published: 03/15/2007
• Est. Priority Date: 09/12/2005
• Status: Abandoned Application

• Current Assignee: Emotiv Systems Pty., Ltd.
• Original Assignee: Emotiv Systems Pty., Ltd.
• Inventors: Thie, Johnson, Delic, Emir, Do, Nam, Pham, Hai, King, William, Le, Tan Thi
• Application Number:
  US11/225,598
• Publication Number:
  US 20070060830A1
• Time in Patent Office: Days
• US Class Current: 600/544
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/7203 for noise prevention, reduction or removal
• CPC Class Codes: A61B 5/7239 using differentiation including higher order derivatives
• CPC Class Codes: A61B 5/726 using Wavelet transforms
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

A method of detecting and classifying facial muscle movements, comprising the steps of: detecting bio-signals from a plurality of scalp electrodes; and applying one or more than one facial muscle movement-detection algorithm to a portion of the bio-signals affected by a predefined type of facial muscle movement in order to detect facial muscle movements of that predefined type.

Claims

1. A method of detecting and classifying facial muscle movements, comprising the steps of:

a) detecting bio-signals from one or more than one bio-signal detector; and

b) applying one or more than one facial muscle movement-detection algorithm to a portion of the bio-signals affected by a predefined type of facial muscle movement in order to detect the facial muscle movements of the predefined type.

2. The method according to claim 1, wherein the step of applying one or more than one facial muscle movement-detection algorithm to the bio-signals comprises comparing the bio-signal portion to a signature defining one or more than one distinctive signal characteristics of the predefined facial muscle movement type.

3. The method according to claim 2, wherein the step of applying one or more than one facial muscle movement-detection algorithm to the bio-signals comprises directly comparing bio-signals from one or more than one predetermined bio-signal detectors to that signature.

4. The method according to claim 2, wherein the step of applying one or more than one facial muscle movement-detection algorithm to the bio-signals comprises:

a) projecting bio-signals from the plurality of bio-signal detectors on one or more than one predetermined component vectors; and

b) comparing the projection of the bio-signals onto one or more than one component vectors to that signature.

5. The method according to claim 4, further comprising applying a desired transform to the projected bio-signal after the projection of the bio-signals from the plurality of detectors on one or more than one component vectors, and before the projected bio-signal is compared to that signature.

6. The method according to claim 4, wherein the predetermined component vectors are determined from applying a first component analysis to historically collected bio-signals generated during facial muscle movement types of the type corresponding to that signature.

7. The method according to claim 6, wherein the first component analysis applied to the historically collected bio-signals is independent component analysis (ICA).

8. The method according to claim 6, wherein the first component analysis applied to the historically collected bio-signals is principal component analysis (PCA).

9. The method according to claim 4, wherein the one or more than one component vectors are updated during facial muscle movement-detection and classification.

10. The method according to claim 2, further comprising updating the signature during the course of facial muscle movement-detection and classification.

11. The method according to claim 10, wherein the signature is updated by changing thresholds forming at least part of the distinctive signal characteristics of the signature.

12. The method according to claim 2, wherein the step of applying one or more than one facial muscle movement-detection algorithm to the bio-signals comprises:

a) applying a desired transform to the bio-signals; and

b) comparing the results of the desired transform to that signature.

13. The method according to claims 12, wherein the transform is one or more than one transform selected from the group consisting of a Fourier transform and a wavelet transform.

14. A method according to claim 4 further comprising:

a) applying a second component analysis to the detected bio-signals; and

b) using the results of the second component analysis to update the one or more than one predetermined component vectors during bio-signal detection.

15. The method according to claim 14, wherein the second component analysis is principal component analysis (PCA).

16. The method according to claim 1, wherein the step of applying one or more than one facial muscle movement-detection algorithm to the bio-signals comprises separating the bio-signals resulting from the predefined type of facial muscle movement from one or more than one sources of noise in the bio-signals.

17. The method according to claim 16, wherein the sources of noise comprise one or more than one source selected from the group consisting of electromagnetic interference (EMI), and bio-signals not resulting from the predefined type of facial muscle movement.

18. The method according to claim 1, wherein the facial muscle movement type is one or more than one facial muscle movement type selected from the group consisting of blinking, winking, frowning, smiling and laughing.

19. The method according to claim 1, wherein the facial muscle movement type is one or more than one facial muscle movement type selected from the group consisting of eye-movements, yawning, chewing and talking.

20. The method according to claim 1, wherein the bio-signals comprise electroencephalograph (EEG) signals.

21. The method according to claim 1, further comprising generating a control signal representative of the detected facial muscle movement type for input to a gaming application.

22. An apparatus for detecting and classifying facial muscle movements, comprising:

a) a sensor interface for receiving bio-signals from one or more than one bio-signal detector; and

b) a processing system for carrying out the step of applying one or more than one facial muscle movement-detection algorithm to a portion of the bio-signals affected by a predefined type of facial muscle movement in order to detect facial muscle movements of that predefined type.

23. The apparatus according to claim 22, wherein the processing system compares the bio-signal portion to a signature defining one or more than one distinctive signal characteristics of the predefined facial muscle movement type.

24. The apparatus according to claim 23, wherein the processing system directly compares bio-signals from one or more than one predetermined bio-signal detectors to that signature.

25. The apparatus according to claim 23, wherein the processing system projects bio-signals from the plurality of bio-signal detectors on one or more than one predetermined component vectors;

and then compares the projection of the bio-signals onto one or more than one component vectors to that signature.

26. The apparatus according to claim 25, wherein after the projection of the bio-signals from the plurality of detectors on one or more than one component vectors and before the projected bio-signal is compared to that signature;

a desired transform is applied to the projected bio-signal.

27. The apparatus according to claim 25, wherein the predetermined component vectors are determined from applying a first component analysis to historically collected bio-signals generated during facial muscle movement types of the type corresponding to that signature.

28. The apparatus according to claim 27, wherein the first component analysis applied to the historically collected bio-signals is independent component analysis (ICA).

29. The apparatus according to claim 27, wherein the first component analysis applied to the historically collected bio-signals is principal component analysis (PCA).

30. The apparatus according to claim 25, wherein the one or more than one component vectors are updated during facial muscle movement-detection and classification.

31. The apparatus according to claim 23, wherein the signature is updated during the course of facial muscle movement-detection and classification.

32. The apparatus according to claim 31, wherein the signature is updated by changing thresholds forming at least part of the distinctive signal characteristics of the signature.

33. The apparatus according to claim 23, wherein the processing system applies a desired transform to the bio-signals;

and compares the results of the desired transform to that signature.

34. The apparatus according to claims 33, wherein the transform is selected from one or more than one transform selected from the group consisting of a Fourier transform and a wavelet transform.

35. A method according to claim 25, wherein the processing system applies a second component analysis to the detected bio-signals, and uses the results of the second component analysis to update the one or more than one predetermined component vectors during bio-signal detection.

36. The apparatus according to claim 35, wherein the second component analysis is principal component analysis (PCA).

37. The apparatus according to claim 22, wherein the processing system separates the bio-signals resulting from the predefined type of facial muscle movement from one or more than one sources of noise in the bio-signals.

38. The apparatus according to claim 37, wherein the sources of noise comprise one or more than one selected from the group consisting of electromagnetic interference (EMI), and bio-signals not resulting from the predefined type of facial muscle movement.

39. The apparatus according to claim 22, wherein the facial muscle movement types comprise one or more than one facial expression selected from the group consisting of blinking, winking, frowning, smiling and laughing.

40. The apparatus according to claim 22, wherein facial muscle movement types comprise one or more than one facial expression selected from the group consisting of eye-movements, yawning, chewing and talking.

41. The apparatus according to claim 22, wherein the bio-signals comprise electroencephalograph (EEG) signals.

42. The apparatus according to claim 22, wherein the processing system generates a control signal representative of the detected facial muscle movement type for input to a gaming application.