Human-machine interface based on task-specific temporal postural synergies

• US 9,034,055 B2
• Filed: 09/27/2011
• Issued: 05/19/2015
• Est. Priority Date: 09/29/2010
• Status: Active Grant

• Current Assignee: University of Pittsburgh of The Commonwealth System of Higher Education
• Original Assignee: University of Pittsburgh of The Commonwealth System of Higher Education
• Inventors: Vinjamuri, Ramana Kumar, Wang, Wei, Mao, Zhi-Hong, Weber, Douglas John
• Primary Examiner(s): Woznicki, Jacqueline
• Application Number:
  US13/246,190
• Publication Number:
  US 20120078381A1
• Time in Patent Office: 1,330 Days
• Field of Search: 623/24, 700/56, 700/245, 600/544, 706/22, 706/53, 414/1
• US Class Current: 623/24
• CPC Class Codes: A61B 2505/09 Rehabilitation or training
• CPC Class Codes: A61B 2562/046 in a matrix array
• CPC Class Codes: A61B 5/1107 Measuring contraction of parts of the body, e.g. organ, muscle apparatus for measuring work or force in general G01L
• CPC Class Codes: A61B 5/24 Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
• CPC Class Codes: A61B 5/6806 Gloves
• CPC Class Codes: A61F 2/583 Hands; Wrist joints
• CPC Class Codes: A61F 2/72 Bioelectric control, e.g. myoelectric
• CPC Class Codes: A61F 2002/6827 Feedback system for providing user sensation, e.g. by force, contact or position
• CPC Class Codes: A61F 2002/704 computer-controlled, e.g. robotic control
• CPC Class Codes: A61F 2002/7615 Measuring means for implanting artificial joints A61F2/4657
• CPC Class Codes: A61F 2002/7625 for measuring angular position
• CPC Class Codes: A61F 2002/763 for measuring spatial position, e.g. global positioning system [GPS]
• CPC Class Codes: A61F 2002/7635 for measuring force, pressure or mechanical tension
• CPC Class Codes: A61F 2002/764 for measuring acceleration
• CPC Class Codes: A61F 2002/7665 for measuring temperatures
• CPC Class Codes: A61H 2230/08 Other bio-electrical signals
• CPC Class Codes: A61H 2230/085 used as a control parameter for the apparatus
• CPC Class Codes: A61H 2230/10 Electroencephalographic signals
• CPC Class Codes: A61H 2230/105 used as a control parameter for the apparatus
• CPC Class Codes: A61H 2230/60 Muscle strain, i.e. measured on the user, e.g. Electromyography [EMG]
• CPC Class Codes: A61H 2230/605 : used as a control parameter for the apparatus
• CPC Class Codes: G05B 2219/40195 : Tele-operation, computer assisted manual operation

Abstract

A synergy-based human-machine interface that uses low-dimensional command signals to control a high dimensional virtual, robotic or paralyzed human hand is provided. Temporal postural synergies are extracted from angular velocities of finger joints of five healthy subjects when they perform hand movements that are similar to activities of daily living. Extracted Synergies are used in real-time brain control, where a virtual, robotic or paralyzed human hand is controlled to manipulate virtual or real world objects.

Claims

1. A method of controlling movement of an extremity of a primate, robot, or machine, wherein the extremity comprises a controlled plurality of joints, comprising:

recording a plurality of tasks performed by a model subject via a movement recording component having a model plurality of joints;

extracting a plurality of temporal postural synergies from angular velocities of the model plurality of joints based on the performance of the plurality of tasks, wherein the plurality of temporal postural synergies comprises common movement primitives associated with the plurality of tasks;

receiving a plurality of control signals from a controlling subject with a neurological disorder;

controlling a subset of the plurality of temporal postural synergies with the plurality of control signals, wherein the plurality of control signals comprises one or more of a neurological signal, a muscular signal, or a signal received via a peripheral sensor; and

controlling the movement of the extremity via calculating positions of the controlled plurality of joints, wherein calculating positions of the controlled plurality of joints comprises a processor performing the following steps;

convolving a subset of the plurality of control signals and the subset of temporal postural synergies with a plurality of finite impulse filters;

adding an output response from the plurality of finite impulse filters that obtains a resultant angular velocity profile; and

integrating the resultant angular velocity profile to obtain the calculated positions of the controlled plurality of joints.

2. The method of claim 1, wherein extracting a plurality of temporal postural synergies from angular velocities of the model plurality of joints based on the performance of the plurality of tasks comprises:

determining a joint angle for each of the model plurality of joints based on a result of the plurality of tasks;

calculating the angular velocities of the model plurality of joints based on the joint angle;

deriving a plurality of kinematic synergies from the angular velocities; and

deriving the plurality of temporal postural synergies from a subset of kinematic synergies from a digital library by integrating angular velocity profiles of the subset of kinematic synergies.

3. The method of claim 2, wherein prior to deriving the plurality of temporal postural synergies from a subset of kinematic synergies by integrating angular velocity profiles of the subset of kinematic synergies, the method further comprises:

forming the digital library of kinematic synergies by storing the plurality of kinematic synergies; and

retrieving the subset of kinematic synergies from the digital library of kinematic synergies.

4. The method of claim 3, wherein the plurality of kinematic synergies are derived using linear and/or non-linear dimensionality reduction methods.

5. The method of claim 1, wherein the plurality of temporal postural synergies are synchronous synergies and are extracted from the model subjects, the model subjects having no neurological disorders.

6. The method of claim 5, wherein the angular velocity profile of the movement of the extremity is a weighted linear combination of the synchronous synergies extracted from recordings where impulses of the synchronous synergies occur at the same time.

7. The method of claim 6, wherein the control signals are invasive or non-invasive brain signals and/or muscular signals and/or signals detected by peripheral sensors.

8. The method of claim 7, wherein extracting a plurality of temporal postural synergies from angular velocities of a model plurality of joints based on the performance of the plurality of tasks comprises:

determining a joint angle for each of the model plurality of joints based on a result of the plurality of tasks;

calculating the angular velocities of the model plurality of joints based on the joint angle;

deriving kinematic synergies from the angular velocities; and

deriving the temporal postural synergies from a subset of kinematic synergies from a digital library by integrating angular velocity profiles of the subset kinematic synergies.

9. The method of claim 8, wherein prior to deriving the plurality of temporal postural synergies from the subset of kinematic synergies by integrating angular velocity profiles of the subset of kinematic synergies, the method further comprises:

storing the plurality of kinematic synergies thereby forming the digital library of kinematic synergies; and

retrieving the subset of kinematic synergies from the digital library of kinematic synergies.

10. The method of claim 9, wherein the plurality of kinematic synergies are derived using linear and/or non-linear dimensionality reduction methods.

11. The method of claim 1, wherein the plurality of temporal postural synergies are asynchronous synergies extracted from the model subjects using linear and/or nonlinear dimensionality reduction methods, the model subjects having no neurological disorders.

12. The method of claim 11, wherein the angular velocity profile of the movement of the extremity is a weighted linear combination of the asynchronous synergies extracted from recordings where impulses of the asynchronous synergies occur at different times.

13. The method of claim 12 further comprising varying a weight of the asynchronous synergy and a time shift of the asynchronous synergy according to a type of task.

14. The method of claim 13, wherein the control signals are invasive or non-invasive brain signals and/or muscular signals and/or signals detected by peripheral sensors.