HUMAN-MACHINE INTERFACE BASED ON TASK-SPECIFIC TEMPORAL POSTURAL SYNERGIES

• US 20120078381A1
• Filed: 09/27/2011
• Published: 03/29/2012
• Est. Priority Date: 09/29/2010
• Status: Active Grant

• Current Assignee: University of Pittsburgh of The Commonwealth System of Higher Education
• Original Assignee: University of Pittsburgh of The Commonwealth System of Higher Education
• Inventors: Vinjamuri, Ramana Kumar, Wang, Wei, Mao, Zhi-Hong, Weber, Douglas John
• Granted Patent: US 9,034,055 B2
• Time in Patent Office: Days
• US Class Current: 623/25
• CPC Class Codes: A61B 2505/09 Rehabilitation or training
• CPC Class Codes: A61B 2562/046 in a matrix array
• CPC Class Codes: A61B 5/1107 Measuring contraction of parts of the body, e.g. organ, muscle apparatus for measuring work or force in general G01L
• CPC Class Codes: A61B 5/24 Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
• CPC Class Codes: A61B 5/6806 Gloves
• CPC Class Codes: A61F 2/583 Hands; Wrist joints
• CPC Class Codes: A61F 2/72 Bioelectric control, e.g. myoelectric
• CPC Class Codes: A61F 2002/6827 Feedback system for providing user sensation, e.g. by force, contact or position
• CPC Class Codes: A61F 2002/704 computer-controlled, e.g. robotic control
• CPC Class Codes: A61F 2002/7615 Measuring means for implanting artificial joints A61F2/4657
• CPC Class Codes: A61F 2002/7625 for measuring angular position
• CPC Class Codes: A61F 2002/763 for measuring spatial position, e.g. global positioning system [GPS]
• CPC Class Codes: A61F 2002/7635 for measuring force, pressure or mechanical tension
• CPC Class Codes: A61F 2002/764 for measuring acceleration
• CPC Class Codes: A61F 2002/7665 for measuring temperatures
• CPC Class Codes: A61H 2230/08 Other bio-electrical signals
• CPC Class Codes: A61H 2230/085 used as a control parameter for the apparatus
• CPC Class Codes: A61H 2230/10 Electroencephalographic signals
• CPC Class Codes: A61H 2230/105 used as a control parameter for the apparatus
• CPC Class Codes: A61H 2230/60 Muscle strain, i.e. measured on the user, e.g. Electromyography [EMG]
• CPC Class Codes: A61H 2230/605 : used as a control parameter for the apparatus
• CPC Class Codes: G05B 2219/40195 : Tele-operation, computer assisted manual operation

Abstract

A synergy-based human-machine interface that uses low-dimensional command signals to control a high dimensional virtual, robotic or paralyzed human hand is provided. Temporal postural synergies are extracted from angular velocities of finger joints of five healthy subjects when they perform hand movements that are similar to activities of daily living. Extracted Synergies are used in real-time brain control, where a virtual, robotic or paralyzed human hand is controlled to manipulate virtual or real world objects.

Claims

1. A method of controlling movement of an extremity comprising:

performing a plurality of tasks with a movement recording component having a plurality of joints;

extracting a plurality of temporal postural synergies from angular velocities of the plurality of joints based on the performance of the plurality of tasks;

receiving a plurality of control signals from a subject with a neurological disorder;

controlling the subset of the plurality of temporal postural synergies with the plurality of control signals;

convolving a subset of the plurality of control signals and the subset of temporal postural synergies with a plurality of finite impulse filters;

adding an output response from the plurality of finite impulse filters that obtains a resultant angular velocity profile; and

integrating the resultant angular velocity profile that obtains a joint position used to control movement of the extremity.

2. The method of claim 1, wherein extracting a plurality of temporal postural synergies from angular velocities of the plurality of joints based on the performance of the plurality of tasks comprises:

determining a joint angle for each of the plurality of joints based on a result of the plurality of tasks;

calculating the angular velocities of the plurality of joints based on the joint angle;

deriving a plurality of kinematic synergies from the angular velocities; and

deriving the plurality of temporal postural synergies from a subset of kinematic synergies from a digital library by integrating angular velocity profiles of the subset of kinematic synergies.

3. The method of claim 2, wherein prior to deriving the plurality of temporal postural synergies from a subset of kinematic synergies by integrating angular velocity profiles of the subset of kinematic synergies, the method further comprises:

storing the plurality of kinematic synergies thereby forming the digital library of kinematic synergies; and

retrieving the subset of kinematic synergies from the digital library of kinematic synergies.

4. The method of claim 3, wherein the plurality of kinematic synergies are derived using linear and/or non-linear dimensionality reduction methods.

5. The method of claim 1, wherein the plurality of temporal postural synergies are synchronous synergies and are extracted from subjects having no neurological disorders.

6. The method of claim 5, wherein the angular velocity profile of the movement of the extremity is a weighted linear combination of the synchronous synergies extracted from recordings where impulses of the synchronous synergies occur at the same time.

7. The method of claim 6, wherein the control signals are invasive or non-invasive brain signals and/or muscular signals and/or signals detected by peripheral sensors.

8. The method of claim 7, wherein extracting a plurality of temporal postural synergies from angular velocities of a plurality of joints based on the performance of the plurality of tasks comprises:

determining a joint angle for each of the plurality of joints based on a result of the plurality of tasks;

calculating the angular velocities of the plurality of joints based on the joint angle;

deriving kinematic synergies from the angular velocities; and

deriving the temporal postural synergies from a subset of kinematic synergies from a digital library by integrating angular velocity profiles of the subset kinematic synergies.

9. The method of claim 8, wherein prior to deriving the plurality of temporal postural synergies from the subset of kinematic synergies by integrating angular velocity profiles of the subset of kinematic synergies, the method further comprises:

storing the plurality of kinematic synergies thereby forming the digital library of kinematic synergies; and

retrieving the subset of kinematic synergies from the digital library of kinematic synergies.

10. The method of claim 9, wherein the plurality of kinematic synergies are derived using linear and/or non-linear dimensionality reduction methods.

11. The method of claim 1, wherein the plurality of temporal postural synergies are asynchronous synergies extracted from subjects having no neurological disorders using linear and/or nonlinear dimensionality reduction methods.

12. The method of claim 11, wherein the angular velocity profile of the movement of the extremity is a weighted linear combination of the asynchronous synergies extracted from recordings where impulses of the asynchronous synergies occur at different times.

13. The method of claim 12 further comprising varying a weight of the asynchronous synergy and a time shift of the asynchronous synergy according to a type of task.

14. The method of claim 13, wherein the control signals are invasive or non-invasive brain signals and/or muscular signals and/or signals detected by peripheral sensors.

15. A human-machine interface (HMI) system comprising:

an extraction component that extracts a plurality of kinematic synergies via a movement recording component, wherein the movement recording component includes a plurality of joints;

a transformation component that transforms a subset of kinematic synergies from a digital library into a plurality of temporal postural synergies;

a receiving component that receives control signals from a subject having neurological disorders;

a convolving component that convolves the control signals and the plurality of temporal postural synergies with a plurality of finite impulse filters;

a combination component that combines output responses from the convolving component that obtains a resultant angular velocity profile; and

a calculating component that calculates movement of an extremity based on the resultant angular velocity profile.

16. The HMI system of claim 15 further comprising:

a storage component that stores the plurality of kinematic synergies thereby forming the digital library of kinematic synergies;

a retrieving component that retrieves a subset of kinematic synergies from the digital library of kinematic synergies;

17. The HMI system of claim 15, wherein the plurality of kinematic synergies are synchronous synergies and are extracted from subjects having no neurological disorders using linear and/or nonlinear dimensionality reduction methods.

18. The HMI system of claim 15, wherein the plurality of kinematic synergies are asynchronous synergies and are extracted from subjects having no neurological disorders using linear and/or nonlinear dimensionality reduction methods.

19. The HMI system of claim 15, wherein the extraction component extracts a joint angle for each of the plurality of joints based on a result of a plurality of tasks performed by the subjects having no neurological disorders, wherein angular velocities of the plurality of joints is derived for each joint angle, and wherein the plurality of kinematic synergies is derived from the angular velocities.

20. The HMI system of claim 19, wherein the control signals are invasive or non-invasive brain signals and/or muscular signals and/or signals detected by peripheral sensors.