Motion recognition device and method

• US 10,172,541 B2
• Filed: 02/04/2016
• Issued: 01/08/2019
• Est. Priority Date: 09/25/2015
• Status: Active Grant

• Current Assignee: J-MEX Inc.
• Original Assignee: J-MEX Inc.
• Inventors: Liao, Wen-Hsuan, Chen, Chi-hung, Lee, Meng-Yu, Chen, Chao-Ling, Hsu, Chih-Hung, Hsieh, Chi-Hung, Huang, Chun-Yuan, Hwang, Deng-Huei, Tsai, Kun-Ming, Ni, Tsang-Der, Chen, I-Tang, Tone, Kwang-Sing
• Primary Examiner(s): Dunlap, Jonathan
• Assistant Examiner(s): DAVIS-HOLLINGTON, OCTAVIA L
• Application Number:
  US15/015,575
• Publication Number:
  US 20170086711A1
• Time in Patent Office: 1,069 Days
• Field of Search: 738654
• CPC Class Codes: A61B 2503/10 Athletes
• CPC Class Codes: A61B 2562/04 Arrangements of multiple sensors of the same type
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/1121 Determining geometric values, e.g. centre of rotation or angular range of movement
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/1126 using a particular sensing technique
• CPC Class Codes: A61B 5/681 Wristwatch-type devices
• CPC Class Codes: A63B 2023/0411 Squatting exercises
• CPC Class Codes: A63B 21/0724 Bar-bells; Hand bars
• CPC Class Codes: A63B 21/0726 Dumb bells, i.e. with a central bar to be held by a single hand, and with weights at the ends
• CPC Class Codes: A63B 2220/803 Motion sensors
• CPC Class Codes: A63B 23/0458 Step exercisers without moving parts with movable steps A63B22/04
• CPC Class Codes: A63B 23/1218 Chinning, pull-up, i.e. concentric movement

Abstract

A motion recognition device includes a sensing unit and a processing unit. The sensing unit generates a sense signal in response to a body motion occurring at a specific position on a user'"'"'s body, wherein the sense signal includes a first sense signal portion and a second sense signal portion different from the first sense signal portion, and the body motion belongs to a motion segment of a motion type. The processing unit processes the sense signal to generate a motion parameter signal structure including a fusion signal of the first and the second sense signal portions, and recognizes the specific position to determine an effective reference signal for recognition of the motion type based on the motion parameter signal structure.

Claims

1. A motion recognition device, comprising:

a sensing unit generating a sense signal in response to a first body motion occurring at a specific position out of a plurality of different positions on a user'"'"'s body, wherein the sense signal includes a first sense signal portion and a second sense signal portion different from the first sense signal portion, and the first body motion belongs to a motion segment of a first motion type; and

a processing unit electrically connected to the sensing unit and processing the sense signal to generate a motion parameter signal structure including a fusion signal of the first and the second sense signal portions, and recognizing the specific position on the user'"'"'s body to determine an effective reference signal for recognition of the first motion type based on the motion parameter signal structure, wherein the processing unit generates a specific position code representing the specific position based on the motion parameter signal structure, and thereby recognizes the specific position,

wherein;

the specific position is selected from the plurality of different positions on the user'"'"'s body;

the motion recognition device is further configured to have an orientation, a gravity direction and a body coordinate system used to determine the orientation, and fastened at the specific position;

the sense signal is generated in relation to the body coordinate system;

in a state that the orientation is directed to a predetermined direction in relation to the gravity direction, the processing unit generates the specific position code representing the specific position based on the motion parameter signal structure, and thereby recognizes the specific position;

the predetermined direction is determined based on the specific position;

the predetermined direction and the gravity direction have a first angle therebetween;

the fusion signal is a signal of an estimated angle associated with the first angle;

in this state, the processing unit makes a first decision on whether the processing unit detects a trigger signal; and

when the first decision is positive, the processing unit generates the specific position code based on the estimated angle.

2. The motion recognition device according to claim 1, wherein:

the sensing unit comprises;

an accelerometer providing an accelerometer signal to the processing unit, wherein the accelerometer signal is the first sense signal portion; and

a gyroscope providing a gyroscope signal to the processing unit, wherein the gyroscope signal is the second sense signal portion;

the first motion type is one selected from a plurality of motion types;

the plurality of motion types are predetermined in relation to the specific position, and respectively have a plurality of principal motion axis directions in relation to the body coordinate system, wherein the plurality of principal motion axis directions are detected beforehand to generate a plurality of principal motion axis direction data units respectively corresponding to the plurality of principal motion axis directions;

the processing unit obtains a recognition reference data unit based on the specific position code, wherein;

the recognition reference data unit is predetermined based on the plurality of principal motion axis direction data units, and includes a candidate reference signal code data unit and a motion type indicator data unit;

the candidate reference signal code data unit includes a first candidate reference signal code;

the first candidate reference signal code represents a first candidate reference signal derived from the motion parameter signal structure;

the motion type indicator data unit includes a motion type indicator corresponding to the first candidate reference signal code; and

the motion type indicator indicates one of an invalid motion type and a second motion type included in the plurality of motion types; and

the processing unit determines the first candidate reference signal based on the motion parameter signal structure and the first candidate reference signal code.

3. The motion recognition device according to claim 2, wherein:

the first body motion includes a first motion portion and a second motion portion adjacent to the first motion portion;

the first and the second motion portions respectively form a first motion cycle and a second motion cycle adjacent to the first motion cycle;

the motion parameter signal structure includes a first motion parameter signal structure portion and a second motion parameter signal structure portion respectively corresponding to the first and the second motion portions;

the first candidate reference signal includes a first candidate reference signal portion and a second candidate reference signal portion respectively corresponding to the first and the second motion portions; and

the processing unit obtains a first motion characteristic value data unit from the motion parameter signal structure based on the first candidate reference signal, wherein;

the first motion characteristic value data unit includes a first motion characteristic value data unit portion and a second motion characteristic value data unit portion respectively corresponding to the first and the second motion portions;

the processing unit obtains the first motion characteristic value data unit portion from the first motion parameter signal structure portion based on the first candidate reference signal portion; and

the processing unit obtains the second motion characteristic value data unit portion from the second motion parameter signal structure portion based on the second candidate reference signal portion.

4. The motion recognition device according to claim 3, wherein:

the processing unit generates a difference data unit based on the first and the second motion characteristic value data unit portions, and makes a first decision on whether the difference data unit satisfies a first specific condition for a periodic-motion start decision;

when the first decision is positive, the processing unit recognizes the effective reference signal as the first candidate reference signal, determines that the first body motion satisfies a predetermined periodic-motion start condition, and makes a second decision on whether the motion type indicator indicates one selected from the plurality of motion types based on the motion type indicator;

when the second decision is positive, the processing unit recognizes the first motion type as the second motion type based on the motion type indicator, and obtains a motion type code representing the first motion type;

the motion parameter signal structure includes a plurality of motion parameter signals;

the recognition reference data unit further includes a representative signal code representing a representative signal included in the plurality of motion parameter signals;

the processing unit determines the representative signal based on the motion parameter signal structure and the representative signal code;

the processing unit obtains a representative extreme value deviation between the first and the second motion characteristic value data unit portions by comparing the first motion characteristic value data unit portion with the second motion characteristic value data unit portion;

the first motion characteristic value data unit portion includes a first motion characteristic value data unit sub-portion corresponding to the representative signal, wherein the first motion characteristic value data unit sub-portion has a first maximum value, a first minimum value, and a first difference between the first maximum value and the first minimum value;

the second motion characteristic value data unit portion includes a second motion characteristic value data unit sub-portion corresponding to the representative signal, wherein the second motion characteristic value data unit sub-portion has a second maximum value, a second minimum value, and a second difference between the second maximum value and the second minimum value;

the processing unit obtains a representative value difference from the first and the second differences;

the difference data unit includes the representative extreme value deviation and the representative value difference;

the first specific condition includes a first sub-condition and a second sub-condition;

the first sub-condition is that the representative extreme value deviation falls within a first predetermined value range; and

the second sub-condition is that the representative value difference falls within a second predetermined value range.

5. The motion recognition device according to claim 4, wherein:

the second motion portion is later than the first motion portion;

the second motion characteristic value data unit portion includes a plurality of specific motion characteristic values;

the processing unit provides beforehand a first characteristic function code data unit associated with the first candidate reference signal code and a first plurality of recognition value domains, wherein;

the first characteristic function code data unit is employed to indicate one of the first plurality of recognition value domains including a first recognition value domain, a second recognition value domain adjacent to the first recognition value domain, and a first confidence value domain between the first and the second recognition value domains; and

the first and the second recognition value domains respectively indicate a third motion type and a fourth motion type, each of which is included in the plurality of motion types;

the recognition device establishes the first characteristic function code data unit beforehand by sensing a plurality of body motions occurring at the specific position, wherein the plurality of body motions are divided into a plurality of motion groups respectively belonging to the plurality of motion types;

the first characteristic function code data unit represents a motion type cut function employed to indicate the one of the first plurality of recognition value domains, and is expressed based on a relationship among the plurality of specific motion characteristic values;

when the second decision is negative, the processing unit performs a first calculation to generate a first recognition value based on the first characteristic function code data unit and the second motion characteristic value data unit portion, and makes a third decision on whether the first recognition value belongs to one of the first and the second recognition value domains;

when the third decision is positive, the processing unit determines an effective recognition value domain in the first and the second recognition value domains that the first recognition value belongs to, recognizes the first motion type as an effective motion type indicated by the effective recognition value domain, and obtains the motion type code corresponding to the effective recognition value domain; and

the processing unit generates a motion measurement information associated with the first body motion based on the motion type code.

6. The motion recognition device according to claim 5, wherein:

the processing unit further provides beforehand a second characteristic function code data unit associated with the first candidate reference signal code and a second plurality of recognition value domains, wherein;

the second characteristic function code data unit is different from the first characteristic function code data unit, and is employed to indicate one of the second plurality of recognition value domains including a third recognition value domain, a fourth recognition value domain adjacent to the third recognition value domain, and a second confidence value domain between the third and the fourth recognition value domains; and

the third and the fourth recognition value domains respectively indicate a fifth motion type and a sixth motion type, each of which is included in the plurality of motion types;

in a second specific condition, one of the third and the fourth recognition value domains at least partially overlaps with one of the first and the second recognition value domains; and

when the third decision is negative, the processing unit performs a second calculation to generate a second recognition value based on the second characteristic function code data unit and the second motion characteristic value data unit portion, and decides whether the second recognition value belongs to one of the third and the fourth recognition value domains, and thereby recognizes the first motion type.

7. The motion recognition device according to claim 3, wherein:

the candidate reference signal code data unit further includes at least a second candidate reference signal code representing at least a second candidate reference signal, wherein;

the second candidate reference signal code represents the second candidate reference signal derived from the motion parameter signal structure; and

the first candidate reference signal and the at least a second candidate reference signal constitute a candidate reference signal combination;

the processing unit determines the at least a second candidate reference signal based on the motion parameter signal structure and the at least a second candidate reference signal code, and obtains at least a second motion characteristic value data unit from the motion parameter signal structure based on the at least a second candidate reference signal, wherein the at least a second motion characteristic value data unit corresponds to the at least a second candidate reference signal; and

when the processing unit processes the first motion characteristic value data unit, the processing unit processes the at least a second motion characteristic value data unit to decide whether the candidate reference signal combination includes the effective reference signal.

8. A motion recognition device, comprising:

a signal generating unit generating a motion parameter signal structure including a fusion signal in response to a body motion occurring at a specific position out of a plurality of different positions on a user'"'"'s body, wherein the body motion belongs to a first motion type; and

a processing unit electrically connected to the signal generating unit and recognizing the specific position on the user'"'"'s body to determine an effective reference signal for recognition of the first motion type based on the motion parameter signal structure, wherein the processing unit generates a specific position code representing the specific position based on the motion parameter signal structure, and thereby recognizes the specific position,

wherein;

the specific position is selected from the plurality of different positions on the user'"'"'s body;

the motion recognition device is further configured to have an orientation, a gravity direction and a body coordinate system used to determine the orientation, and fastened at the specific position;

in a state that the orientation is directed to a predetermined direction in relation to the gravity direction, the processing unit generates the specific position code representing the specific position based on the motion parameter signal structure, and thereby recognizes the specific position;

the predetermined direction is determined based on the specific position;

the predetermined direction and the gravity direction have a first angle therebetween;

the fusion signal is a signal of an estimated angle associated with the first angle;

in this state, the processing unit makes a first decision on whether the processing unit detects a trigger signal; and

when the first decision is positive, the processing unit generates the specific position code based on the estimated angle.

9. The motion recognition device according to claim 8, wherein:

the specific position is selected from a plurality of different positions on the user'"'"'s body;

the signal generating unit comprises a sensing unit and a converting unit coupled to the sensing unit;

the sensing unit generates a sense signal in response to the body motion, and comprises;

an accelerometer providing an accelerometer signal to the processing unit; and

a gyroscope providing a gyroscope signal to the processing unit;

the converting unit generates the motion parameter signal structure in response to the sense signal, wherein the motion parameter signal structure includes a fusion signal of the accelerometer signal and the gyroscope signal;

the processing unit is coupled to the signal generating unit;

the first motion type is one selected from a plurality of motion types;

the plurality of motion types are predetermined in relation to the specific position, and respectively have a plurality of principal motion axis directions in relation to the body coordinate system, wherein the plurality of principal motion axis directions are detected beforehand to generate a plurality of principal motion axis direction data units respectively corresponding to the plurality of principal motion axis directions;

the motion parameter signal structure is generated in relation to the body coordinate system;

the processing unit obtains a recognition reference data unit based on the specific position code, wherein;

the recognition reference data unit is predetermined based on the plurality of principal motion axis direction data units, and includes a candidate reference signal code, and a motion type indicator corresponding to the candidate reference signal code;

the candidate reference signal code represents a candidate reference signal derived from the motion parameter signal structure; and

the motion type indicator indicates one of an invalid motion type and a second motion type included in the plurality of motion types; and

the processing unit determines the candidate reference signal based on the motion parameter signal structure and the candidate reference signal code.

10. The motion recognition device according to claim 9, wherein:

the body motion includes a first motion portion and a second motion portion adjacent to the first motion portion;

the first and the second motion portions respectively form a first motion cycle and a second motion cycle adjacent to the first motion cycle;

the processing unit obtains a motion characteristic value data unit from the motion parameter signal structure based on the candidate reference signal, wherein the motion characteristic value data unit includes a first motion characteristic value data unit portion and a second motion characteristic value data unit portion respectively corresponding to the first and the second motion portions;

the processing unit generates a difference data unit based on the first and the second motion characteristic value data unit portions;

the processing unit makes a first decision on whether the difference data unit satisfies a first specific condition for a periodic-motion start decision;

when the first decision is positive, the processing unit recognizes the effective reference signal as the candidate reference signal, and based on the motion type indicator, makes a second decision on whether the motion type indicator indicates one selected from the plurality of motion types; and

when the second decision is positive, the processing unit recognizes the first motion type as the second motion type based on the motion type indicator, and obtains a motion type code representing the first motion type.

11. The motion recognition device according to claim 10, wherein:

the second motion portion is later than the first motion portion;

the second motion characteristic value data unit portion includes a plurality of specific motion characteristic values;

the processing unit provides beforehand a characteristic function code data unit expressed based on a relationship among the plurality of specific motion characteristic values, wherein;

the characteristic function code data unit is associated with the candidate reference signal code, a first recognition value domain, and a second recognition value domain adjacent to the first recognition value domain; and

the first and the second recognition value domains respectively indicate a third motion type and a fourth motion type, each of which is included in the plurality of motion types; and

when the second decision is negative, the processing unit performs a calculation to generate a recognition value based on the characteristic function code data unit and the second motion characteristic value data unit portion, determines an effective recognition value domain in the first and the second recognition value domains that the recognition value belongs to, recognizes the first motion type as an effective motion type indicated by the effective recognition value domain, and obtains the motion type code corresponding to the effective recognition value domain.

12. A motion recognition method, comprising steps of:

generating a motion parameter signal structure in response to a first body motion occurring at a specific position on a user'"'"'s body, wherein the first body motion belongs to a first motion type;

generating a specific position code representing the specific position on the user'"'"'s body based on the motion parameter signal structure;

recognizing the specific position to determine an effective reference signal for recognition of the first motion type; and

providing a recognition device, wherein the recognition device is configured to have an orientation, a gravity direction and a body coordinate system used to determine the orientation, and is fastened at the specific position, wherein;

the specific position is selected from a plurality of different positions on the user'"'"'s body;

the step of generating the motion parameter signal structure in response to the first body motion further comprises sub-steps of;

generating a sense signal associated with the body coordinate system in response to the first body motion, wherein the sense signal includes an accelerometer signal and a gyroscope signal; and

processing the sense signal to generate the motion parameter signal structure including a fusion signal of the accelerometer signal and the gyroscope signal;

the predetermined direction is determined based on the specific position;

the predetermined direction and the gravity direction have a first angle therebetween;

the fusion signal is a signal of an estimated angle associated with the first angle; and

the step of generating the specific position code comprises sub-steps of;

in this state, making a first decision on whether a trigger signal is detected; and

when the first decision is positive, generating the specific position code based on the estimated angle.

13. The motion recognition method according to claim 12,

the first motion type is one selected from a plurality of motion types;

the plurality of motion types are predetermined in relation to the specific position, and respectively have a plurality of principal motion axis directions in relation to the body coordinate system, wherein the plurality of principal motion axis directions are detected beforehand to generate a plurality of principal motion axis direction data units respectively corresponding to the plurality of principal motion axis directions; and

obtaining a recognition reference data unit based on the specific position code, wherein;

the recognition reference data unit is predetermined based on the plurality of principal motion axis direction data units, and includes a candidate reference signal code data unit and a motion type indicator data unit;

the candidate reference signal code data unit includes a first candidate reference signal code representing a first candidate reference signal derived from the motion parameter signal structure;

the motion type indicator data unit includes a motion type indicator corresponding to the first candidate reference signal code; and

the motion type indicator indicates one of an invalid motion type and a second motion type included in the plurality of motion types; and

determining the first candidate reference signal based on the motion parameter signal structure and the first candidate reference signal code.

14. The motion recognition method according to claim 13, wherein:

the motion parameter signal structure includes a plurality of motion parameter signals;

the first body motion includes a first motion portion and a second motion portion adjacent to the first motion portion;

the first and the second motion portions respectively form a first motion cycle and a second motion cycle adjacent to the first motion cycle;

the motion parameter signal structure includes a first motion parameter signal structure portion and a second motion parameter signal structure portion respectively corresponding to the first and the second motion portions;

the first candidate reference signal includes a first candidate reference signal portion and a second candidate reference signal portion respectively corresponding to the first and the second motion portions;

the recognition reference data unit further includes a representative signal code representing a representative signal included in the plurality of motion parameter signals; and

the recognition method further comprises steps of;

obtaining a first motion characteristic value data unit from the motion parameter signal structure based on the first candidate reference signal, wherein;

the first motion characteristic value data unit includes a first motion characteristic value data unit portion and a second motion characteristic value data unit portion respectively corresponding to the first and the second motion portions; and

the step of obtaining the first motion characteristic value data unit further comprises sub-steps of;

obtaining the first motion characteristic value data unit portion from the first motion parameter signal structure portion based on the first candidate reference signal portion; and

obtaining the second motion characteristic value data unit portion from the second motion parameter signal structure portion based on the second candidate reference signal portion;

generating a difference data unit based on the first and the second motion characteristic value data unit portions;

making a first decision on whether the difference data unit satisfies a first specific condition for a periodic-motion start decision;

when the first decision is positive, recognizing the effective reference signal as the first candidate reference signal, determining that the first body motion satisfies a predetermined periodic-motion start condition, and based on the motion type indicator, making a second decision on whether the motion type indicator indicates one selected from the plurality of motion types; and

when the second decision is positive, based on the motion type indicator, recognizing the first motion type as the second motion type, and obtaining a motion type code representing the first motion type.

15. The motion recognition method according to claim 14, wherein:

the step of generating the difference data unit further comprises sub-steps of;

obtaining a representative extreme value deviation between the first and the second motion characteristic value data unit portions by comparing the first motion characteristic value data unit portion with the second motion characteristic value data unit portion;

determining the representative signal based on the motion parameter signal structure and the representative signal code, wherein;

the first motion characteristic value data unit portion includes a first motion characteristic value data unit sub-portion corresponding to the representative signal, and the first motion characteristic value data unit sub-portion has a first maximum value, a first minimum value, and a first difference between the first maximum value and the first minimum value; and

the second motion characteristic value data unit portion includes a second motion characteristic value data unit sub-portion corresponding to the representative signal, and the second motion characteristic value data unit sub-portion has a second maximum value, a second minimum value, and a second difference between the second maximum value and the second minimum value; and

obtaining a representative value difference from the first and the second differences, wherein the difference data unit includes the representative extreme value deviation and the representative value difference;

the first specific condition includes a first sub-condition and a second sub-condition;

the first sub-condition is that the representative extreme value deviation falls within a first predetermined value range; and

the second sub-condition is that the representative value difference falls within a second predetermined value range.

16. The motion recognition method according to claim 15, further comprising steps of:

providing a first characteristic function code data unit beforehand by sensing a plurality of body motions occurring at the specific position, wherein;

the second motion portion is later than the first motion portion;

the second motion characteristic value data unit portion includes a plurality of specific motion characteristic values;

the first characteristic function code data unit is associated with the first candidate reference signal code and a first plurality of recognition value domains;

the first characteristic function code data unit is employed to indicate one of the first plurality of recognition value domains;

the first plurality of recognition value domains include a first recognition value domain, a second recognition value domain adjacent to the first recognition value domain, and a first confidence value domain between the first and the second recognition value domains;

the first and the second recognition value domains respectively indicate a third motion type and a fourth motion type, each of which is included in the plurality of motion types;

the plurality of body motions are divided into a plurality of motion groups respectively belonging to the plurality of motion types; and

the first characteristic function code data unit represent a motion type cut function employed to indicate the one of the first plurality of recognition value domains, and is expressed based on a relationship among the plurality of specific motion characteristic values;

when the second decision is negative, performing a first calculation to generate a first recognition value based on the first characteristic function code data unit and the second motion characteristic value data unit portion, and making a third decision on whether the first recognition value belongs to one of the first and the second recognition value domains;

when the third decision is positive, determining an effective recognition value domain in the first and the second recognition value domains that the first recognition value belongs to, recognizing the first motion type as an effective motion type indicated by the effective recognition value domain, and obtaining the motion type code corresponding to the effective recognition value domain; and

generating a motion measurement information associated with the first body motion based on the motion type code.

17. The motion recognition method according to claim 16, further comprising steps of:

providing beforehand a second characteristic function code data unit associated with the first candidate reference signal code and a second plurality of recognition value domains, wherein;

the second characteristic function code data unit is different from the first characteristic function code data unit, and is employed to indicate one of the second plurality of recognition value domains;

the second plurality of recognition value domains include a third recognition value domain, a fourth recognition value domain adjacent to the third recognition value domain, and a second confidence value domain between the third and the fourth recognition value domains;

the third and the fourth recognition value domains respectively indicate a fifth motion type and a sixth motion type, each of which is included in the plurality of motion types; and

in a second specific condition, one of the third and the fourth recognition value domains at least partially overlaps with one of the first and the second recognition value domains; and

when the third decision is negative, performing a second calculation to generate a second recognition value based on the second characteristic function code data unit and the second motion characteristic value data unit portion, and deciding whether the second recognition value belongs to one of the third and the fourth recognition value domains, thereby recognizing the first motion type.

18. The motion recognition method according to claim 14, wherein:

the candidate reference signal code data unit further includes at least a second candidate reference signal code representing at least a second candidate reference signal, wherein;

the second candidate reference signal code represents the second candidate reference signal derived from the motion parameter signal structure; and

the first candidate reference signal and the at least a second candidate reference signal constitute a candidate reference signal combination; and

the recognition method further comprises steps of;

determining the at least a second candidate reference signal based on the motion parameter signal structure and the at least a second candidate reference signal code;

obtaining at least a second motion characteristic value data unit from the motion parameter signal structure based on the at least a second candidate reference signal, wherein the at least a second motion characteristic value data unit corresponds to the at least a second candidate reference signal; and

when the first motion characteristic value data unit is processed, processing the at least a second motion characteristic value data unit to decide whether the candidate reference signal combination includes the effective reference signal.