Musculoskeletal activity recognition system and method

• US 10,130,298 B2
• Filed: 03/15/2013
• Issued: 11/20/2018
• Est. Priority Date: 04/03/2012
• Status: Active Grant

• Current Assignee: Carnegie Mellon University
• Original Assignee: Carnegie Mellon University
• Inventors: Mokaya, Frank, Zhang, Pei
• Primary Examiner(s): Cerioni, Daniel
• Assistant Examiner(s): Warsi, Yasmeen S
• Application Number:
  US14/390,419
• Publication Number:
  US 20150045699A1
• Time in Patent Office: 2,076 Days
• Field of Search: 600587, 600595, 600554
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/0004 characterised by the type of physiological signal transmitted
• CPC Class Codes: A61B 5/0024 for multiple sensor units attached to the patient, e.g. using a body or personal area network
• CPC Class Codes: A61B 5/1101 Detecting tremor
• CPC Class Codes: A61B 5/1107 Measuring contraction of parts of the body, e.g. organ, muscle apparatus for measuring work or force in general G01L
• CPC Class Codes: A61B 5/1126 using a particular sensing technique
• CPC Class Codes: A61B 5/224 Measuring muscular strength
• CPC Class Codes: A61B 5/4519 Muscles A61B5/389, A61B5/224 take precedence
• CPC Class Codes: A61B 5/7203 for noise prevention, reduction or removal
• CPC Class Codes: A61B 5/725 using specific filters therefor, e.g. Kalman or adaptive filters A61B5/316 take precedence
• CPC Class Codes: A61B 5/7257 using Fourier transforms
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: A61B 5/7278 Artificial waveform generation or derivation, e.g. synthesising signals from measured signals A61B5/327 takes precedence
• CPC Class Codes: A61B 5/7425 Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image geometric transforms for image registration G06T3/14; enhancement using image fusion G06T5/50; algorithms for registration of images G06T7/30

Abstract

A muscle activity and skeletal monitoring system and method are disclosed. A network of at least two distributed inertial sensor nodes are configured to communicate with each other over a first interface. Each of the inertial sensor nodes comprises at least one sensor configured to sense muscle vibrations and monitor body motion. A muscle activity recognition and motion tracking and visualization methods also are disclosed.

Claims

1. A muscle activity recognition method, comprising:

detecting, via a network of more than one inertial sensor node distributed on a physically active muscle group of a user'"'"'s body, muscle vibration data, wherein each inertial sensor node comprises at least one inertial sensor including an accelerometer;

determining, via a server communicatively coupled to each inertial sensor node, a unique vibration signature to identify the muscle group of the user'"'"'s body, wherein the unique vibration signature is determined from a stream of muscle vibration data compiled from each accelerometer of each inertial sensor node, and wherein the server is configured to execute a muscle activity recognition module programmed to;

filter the stream of muscle vibration data through a high pass filter;

extract features from the filtered stream of muscle vibration data by calculating a select set of feature vectors;

combine the select set of feature vectors into a matrix, wherein the matrix constitutes the unique vibration signature associated with the muscle group of the user'"'"'s body; and

identify, via a classifier, the muscle group of the user'"'"'s body corresponding to the determined unique vibration signature based on the calculated select set of feature vectors; and

presenting, via a user interface, a human-like avatar comprising a plurality of visualized body parts, wherein the identified muscle group of the user'"'"'s body is highlighted on its corresponding visualized body part of the human-like avatar to distinguish different physically active muscle groups of the user'"'"'s body and to monitor the identified muscle group of the user'"'"'s body.

2. The muscle activity recognition method of claim 1, wherein the identifying of the muscle group of the user'"'"'s body is based on classifier objects stored on the server.

3. The muscle activity recognition method of claim 1, wherein calculating the select set of feature vectors comprises:

calculating a first number of time domain feature vectors and a second number of frequency domain feature vectors.

4. The muscle activity recognition method of claim 3, wherein each time domain feature vector is calculated using a sliding window technique with either a fixed or a variable overlap on the filtered stream of muscle vibration data.

5. The muscle activity recognition method of claim 4, wherein the sliding window technique with the fixed overlap is used, and wherein each frequency domain feature vector is calculated via a discrete Fourier transform on the filtered stream of muscle vibration data contained in each fixed sliding window.

6. The muscle activity recognition method of claim 1, further comprising:

training the classifier using a supervised machine learning technique.

7. The muscle activity recognition method of claim 1, wherein the classifier comprises a J48 decision tree classifier.

8. The muscle activity recognition method of claim 1, further comprising:

detecting muscle vibration data for a single muscle group during an isolation exercise, wherein the muscle vibration data varies based on a fatigue status of the single muscle group;

determining a fresh vibration signature from muscle vibration data associated with the single muscle group in a fresh state and a fatigued vibration signature from muscle vibration data associated with the single muscle group in a fatigued state; and

training the classifier using a supervised machine learning technique using the fresh vibration signature and the fatigued vibration signature;

wherein the server is configured to execute the muscle activity recognition module further programmed to identify, via the classifier, the muscle group of the user'"'"'s body as fresh or fatigued if the identified muscle group corresponds to the single muscle group and the determined unique vibration signature corresponds to the fresh vibration signature or the fatigued vibration signature respectively; and

wherein the method further comprises highlighting the identified muscle group of the user'"'"'s body on its corresponding visualized body part of the human-like avatar in a first way if the identified muscle group of the user'"'"'s body is fresh and in a second way if the identified muscle group of the user'"'"'s body is fatigued to prevent over-exertion of the identified muscle group of the user'"'"'s body.

9. The muscle activity recognition method of claim 1, further comprising:

detecting, via the network of more than one inertial sensor node distributed on the physically active muscle group of the user'"'"'s body, body motion data, wherein each inertial sensor node comprises a plurality of inertial sensors including the accelerometer, a gyroscope, and a magnetometer; and

tracking, via the server, an orientation of the more than one inertial sensor node over time, wherein the server is configured to execute a motion tracking and visualization module programmed to;

filter a stream of accelerometer body motion data, compiled from each accelerometer of each inertial sensor node, through a complimentary filter functioning as a low-pass filter;

filter a stream of gyroscope body motion data, compiled from each gyroscope of each inertial sensor node, through the complimentary filter functioning as a high-pass filter;

filter a stream of magnetometer body motion data, compiled from each magnetometer of each inertial sensor node, through the complimentary filter functioning as a low-pass filter;

correct drift associated with the filtered stream of accelerometer body motion data based on at least one of the filtered stream of gyroscope body motion data or the filtered stream of magnetometer body motion data;

correct drift associated with the filtered stream of gyroscope body motion data based on at least one of the filtered stream of accelerometer body motion data or the filtered stream of magnetometer body motion data; and

correct drift associated with the filtered stream of magnetometer body motion data based on at least one of the filtered stream of accelerometer body motion data or the filtered stream of gyroscope body motion data

wherein an orientation of the more than one inertial sensor node is determined based on the filtered and corrected stream of accelerometer body motion data, the filtered and corrected stream of gyroscope body motion data, and the filtered and corrected stream of magnetometer body motion data over time.

10. The muscle activity recognition method of claim 9, wherein the server is configured to execute the motion tracking and visualization module further programmed to:

map the more than one inertial sensor node to a part of the user'"'"'s body based on a location of the identified muscle group of the user'"'"'s body, identified via the muscle activity recognition module to, determine a raw orientation of the part of the user'"'"'s body over time; and

generate a revised orientation of the part of the user'"'"'s body over time by filtering the determined raw orientation of the part of the user'"'"'s body over time through a range of motion limiter to restrict the orientation of the part of the user'"'"'s body to within limits of a normal human body.

11. The muscle activity recognition method of claim 10, wherein the server is configured to execute the motion tracking and visualization module further programmed to:

animate the human-like avatar in the user interface, based on the revised orientation of the part of the user'"'"'s body over time, such that the corresponding visualized body part of the human-like avatar moves to mimic actual motions of the part of the user'"'"'s body to analyze motion of the part of the user'"'"'s body including form.

12. The muscle activity recognition method of claim 1, wherein the at least one inertial sensor of each inertial sensor node comprises a triple axis accelerometer, a triple axis gyroscope, and a triple axis magnetometer.

13. The muscle activity recognition method of claim 1, wherein each inertial sensor node further comprises a microcontroller chip coupled to the at least one inertial sensor, and wherein the method further comprises:

controlling, via each microcontroller chip, a sampling of the muscle vibration data by each respective inertial sensor node.

14. The muscle activity recognition method of claim 1, wherein the server is communicatively coupled to each inertial sensor node via a data aggregator, and wherein the method further comprises:

sampling, by the data aggregator, the muscle vibration data from each inertial sensor node;

compiling, by the data aggregator, the sampled muscle vibration data into the stream of muscle vibration data; and

transmitting, by the data aggregator, the stream of muscle vibration data to the server.

15. The muscle activity recognition method of claim 14, wherein the transmitting comprises wirelessly transmitting the stream of muscle vibration data to the server to permit user mobility.