Method, system and apparatus for real-time classification of muscle signals from self-selected intentional movements

• US 8,437,844 B2
• Filed: 08/21/2006
• Issued: 05/07/2013
• Est. Priority Date: 08/21/2006
• Status: Active Grant

• Current Assignee: Holland Bloorview Kids Rehabilitation Hospital
• Original Assignee: Holland Bloorview Kids Rehabilitation Hospital
• Inventors: Chau, Tom Tak Kin, Syed Momen, Kaveh
• Primary Examiner(s): TOWA, RENE T
• Application Number:
  US11/506,786
• Publication Number:
  US 20080058668A1
• Time in Patent Office: 2,451 Days
• Field of Search: 600/587, 600/434, 600/381, 600/595, 600544-546, 623 24- 25, 623/27, 623/58, 623/63, 700/213, 700/250, 706/13, 706/14, 375/316
• US Class Current: 600/546
• CPC Class Codes: A61B 5/6824 Arm or wrist
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: A61F 2/583 Hands; Wrist joints
• CPC Class Codes: A61F 2/68 Operating or control means
• CPC Class Codes: A61F 2/72 Bioelectric control, e.g. myoelectric
• CPC Class Codes: A61F 2002/704 computer-controlled, e.g. robotic control
• CPC Class Codes: A61F 2002/705 Electromagnetic data transfer for implantable prostheses A61F2250/0002
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

A new method, system and apparatus is provided that enables muscle signals that correspond to muscle contractions to be mapped to one or more functions of an electronic device such as a prosthetic device or gaming apparatus. Muscle signals are classified in real-time from self-selected intentional movements. A self-training protocol allows users to select and label their own muscle contractions, and is operable to automatically determine the discernible and repeatable muscle signals generated by the user. A visual display means is used to provide visual feedback to users illustrating the responsiveness of the system to muscle signals generated by the user.

Claims

1. A computer-implemented method for (i) a user to train at least one computer substantially in real-time by performing a series of randomly-performed, self-selected, intentional muscle movements selected by the user, and (ii) the user to label the muscle movements, comprising:

a user actuating a user-controlled device according to a series of any randomly-performed, self-selected, intentional movements selected by the user;

the at least one computer recording and compiling the user'"'"'s muscle signals corresponding to the randomly-performed, self-selected, intentional movements of the device, the at least one computer clustering the compiled muscle signals into at least one cluster of signals;

the at least one computer unsupervised-classifying the at least one cluster of signals as at least one non-predetermined function of the user-controlled device, without reference to predetermined classifications of muscle signals; and

after the unsupervised-classifying, the at least one computer accepting user input to label the classified muscle signals to correspond to the at least one function of the user-controlled apparatus.

2. The method of claim 1, wherein, during the moving, recording, and labelling steps, the user is unsupervised.

3. The method of claim 1, wherein, during the moving, recording, and labelling steps, no skin preparation is applied to the skin of the user.

4. The method of claim 1 wherein the device comprises at least one of a prosthesis, a gaming device, a writing apparatus, and a computer input device.

5. The method of claim 1 wherein the muscle signals are recorded at two or more sites on the body of the user.

6. The method of claim 1 whereby the recording and compiling are accomplished by extracting one or more features of the muscle signals.

7. The method of claim 6, wherein the extraction of the one or more features includes calculating at least one of:

(i) autoregressive (AR) coefficient values, (ii) short-time Fourier transform (STFT) coefficient values, (iii) wavelet transform (WT) coefficient values, (iv) wavelet packet transform (WPT) coefficient values, and (v) stationary wavelet transform (SWT) coefficient values, associated with the muscle signals.

8. The method of claim 6, whereby the one or more features of the muscle signals are extracted using a feature extractor.

9. The method of claim 6 whereby the extraction of the one or more features includes calculating natural logarithms of root-mean-square values associated with the muscle signals.

10. The method of claim 6 wherein the clustering comprises calculating one or more clusters in association with the one or more features.

11. The method of claim 10 wherein the labelling comprises using a visual display to provide the user real-time feedback for the user to label cluster centers.

12. The method of claim 10 whereby the one or more cluster centers are calculated using a fuzzy c-means clustering algorithm.

13. The method of claim 10 further comprising the step of using the clusters to calculate membership values in association with the one or more features, whereby the membership values are used to classify the muscle signals.

14. The method of claim 13 whereby classification of the muscle signals occurs on a real-time basis.

15. The method of claim 13, wherein the clusters comprise cluster centers, and wherein the step of calculating one or more clusters includes the step of calculating one or more cluster centers, and wherein the step of using the clusters includes the step of using the cluster centers.

16. The method of claim 15, wherein the step of calculating the membership values includes the step of calculating a distance from a cluster center.

17. A computer-implemented method for a user to train at least one computer substantially in real-time by performing a series of randomly-performed, self-selected, intentional muscle movements selected by the user, the method comprising:

enabling a user to actuate a device according to non-predetermined, randomly-performed, intentional movements of the user and selected by the user;

recording and compiling the user'"'"'s (i) flexor muscle signals and (ii) extensor muscle signals, corresponding to the non-predetermined, randomly-performed, self-selected, intentional movements;

the at least one computer calculating natural logarithms of root-mean-square values associated with the flexor and extensor muscle signals to yield features of the muscle signals;

the at least one computer clustering the flexor and extensor muscle signal features, thereby defining one or more cluster centers corresponding to the flexor and extensor muscle signal features;

the at least one computer calculating cluster membership values by comparing the flexor and extensor muscle signal features and the cluster centers;

the at least one computer unsupervised-classifying one or more clusters as one or more non-predetermined functions of the device, without reference to predetermined classifications of muscle signals; and

the at least one computer unsupervised-classifying newly-received flexor and extensor muscle signal features according to the membership values.

18. A system for a user to train at least one computer substantially in real-time by performing a series of randomly-performed, self-selected, intentional muscle movements selected by the user, comprising:

a device operable to record a user'"'"'s muscle signals in association with the series of randomly-performed, self-selected, intentional muscle movements selected by the user, the device including or being linked to the at least one computer; and

the at least one computer programmed to cause the at least one computer to compile and classify the muscle signals by;

recording and compiling the user'"'"'s muscle signals in association with the series of randomly-performed, self-selected, intentional muscle movements selected by the user;

mapping the muscle signals to the series of randomly-performed, self-selected, intentional muscle movements selected by the user by clustering the muscle signals in accordance with flexor muscle signals and extensor muscle signals into one or more clusters;

unsupervised-classifying the one or more clusters as one or more non-predetermined functions of the device, without reference to predetermined classifications of muscle signals; and

periodically re-clustering the muscle signals in accordance with previously-received muscle signals and newly-received muscle signals.

19. The system of claim 18 further comprising a visual display means that provides cues to the user to perform random, self-selected, specific intentional movements selected by the user.

20. The system of claim 18 wherein the device is operable to record the muscle signals at two or more sites on the body of the user.

21. The system of claim 18 wherein the system includes a feature extractor that is operable to generate features in association with the muscle signals.

22. The system of claim 21 whereby the computer program is operable on the computer to (i) cluster the features and (ii) classify the muscle signals according to membership values calculated in relation to cluster centers.

23. The system of claim 22 whereby the computer program is operable on the computer to classify the muscle signals over a window equal to or less than 0.3 seconds.

24. A muscle signal activated apparatus for a user to train at least one computer substantially in real-time by performing a series of randomly-performed, self-selected, intentional movements selected by the user, comprising:

a movable portion of the apparatus that is operable to record a user'"'"'s muscle signals in association with the randomly-performed, self-selected, intentional movements of the user;

an electronic device linked to the movable portion, the electronic device being operable to define a calibration mode for enabling the user to calibrate the apparatus, said electronic device being operable to;

provide cues to the user to perform the random, self-selected, intentional movements of the user;

record and compile the user'"'"'s muscle signals in association with the randomly-performed, self-selected, intentional movements;

clustering the compiled muscle signals into at least one cluster of signals; and

unsupervised-classifying the at least one cluster of signals as one or more non-predetermined functions of the movable portion, without reference to predetermined classifications of muscle signals.

25. The apparatus of claim 24 wherein the electronic device includes an input means for providing cues to the user, said input means being selected from the group consisting of:

one or more buttons, a keypad, a touch screen, a voice recognition system, or any combination thereof.

26. The apparatus of claim 24 wherein the electronic device is operable to:

generate features from the muscles signals using a feature extractor means;

cluster the features thereby defining one or more cluster centers in association with the features;

calculate membership values by comparing the features and the cluster centers; and

classify the muscle signals according to the membership values.

27. The apparatus of claim 26 wherein the electronic device includes a feedback means that is operable to enable the user to label the cluster centers according to the randomly-performed, self-selected, intentional movements.

28. The apparatus of claim 26, wherein the electronic device is operable to re-cluster the features based on previously-received muscle signals and newly-received muscle signals.

29. The apparatus as claimed in claim 26, wherein the electronic device includes a controller.

30. The apparatus as claimed in claim 29, wherein the controller includes a control program that cooperates with the controller to perform the operations of the electronic device.

31. A non-transitory computer-readable medium for a user to train at least one computer substantially in real-time by performing a series of randomly-performed, self-selected, intentional muscle movements selected by the user, wherein the computer-readable medium includes instructions operable on the at least one computer to:

provide cues to the user to perform the random, self-selected, intentional movements;

record and compile the muscle signals in association with the randomly-performed, self-selected, intentional movements;

clustering the compiled muscle signals into at least one cluster of signals; and

unsupervised-classifying the one or more clusters as one or more non-predetermined functions of the intentional movements, without reference to predetermined classifications of muscle signals.

32. The computer-readable medium of claim 31, wherein the computer-readable medium further includes instructions operable on the at least one computer to re-cluster the features based on previously-received muscle signals and newly-received muscle signals.

33. The computer-readable medium claimed in claim 31, wherein the computer-readable medium further includes instructions operable on the at least one computer to:

generate features from the muscles signals using a feature extractor means;

cluster the features thereby defining one or more cluster centers in association with the features;

calculate membership values by comparing the features and the cluster centers; and

classify the muscle signals according to the membership values.

34. The computer-readable medium of claim 31 wherein the computer-readable medium further includes instructions operable on the at least one computer to enable the user to label the cluster centers.

35. The computer-readable medium of claim 34 wherein the computer-readable medium further includes instructions operable on the at least one computer to cue the user to perform specific intentional movements corresponding to the labels.