Behavior recognition apparatus

• US 8,682,608 B2
• Filed: 05/27/2010
• Issued: 03/25/2014
• Est. Priority Date: 05/27/2009
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Tsuzuki, Takashi, Yoshioka, Mototaka, Takata, Kazutoyo
• Primary Examiner(s): Hindenburg, Max
• Assistant Examiner(s): DANEGA, RENEE A
• Application Number:
  US13/058,072
• Publication Number:
  US 20110144543A1
• Time in Patent Office: 1,398 Days
• Field of Search: 702/141, 600/595
• US Class Current: 702/141
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/389 Electromyography [EMG]
• CPC Class Codes: A61B 5/4538 Evaluating a particular part of the muscoloskeletal system or a particular medical condition
• CPC Class Codes: A61B 5/4576 Evaluating the shoulder
• CPC Class Codes: A61B 5/6824 Arm or wrist
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A behavior recognition apparatus includes a hand movement observation unit which outputs hand movement data indicating a hand movement made by a user; a velocity variation detection unit which detects a time at which the hand movement matches a predetermined movement pattern; and a feature selection unit which selects the hand movement data outputted at the time detected by the velocity variation detection unit. A standard behavior pattern storage unit stores a movement pattern data sequence represented by a hand movement data sequence, in association with a user behavior; and a behavior recognition unit calculates a degree of similarity between the movement pattern data sequence stored in the standard behavior pattern storage unit and the hand movement data selected by the feature selection unit, and recognizes, as the user behavior, a behavior represented by the movement pattern data sequence when the degree of similarity is the highest.

Claims

1. A behavior recognition apparatus which recognizes a user behavior from a hand movement made by a user, the behavior recognition apparatus comprising:

a hand movement observation unit configured to detect the hand movement made by the user and output hand movement data indicating the hand movement;

a velocity variation detection unit configured to detect a time at which a variation in velocity of a hand of the user occurs, from the hand movement data outputted from the hand movement observation unit;

a feature selection unit configured to select the hand movement data that is outputted from the hand movement observation unit at specific times including the time detected by the velocity variation detection unit and times before and after the detected time;

a standard behavior pattern storage unit configured to store a movement pattern data sequence represented by a hand movement data sequence, in association with the user behavior; and

a behavior recognition unit configured to calculate a degree of similarity between the movement pattern data sequence stored in the standard behavior pattern storage unit and the hand movement data selected by the feature selection unit, and to recognize, as the user behavior, a behavior represented by the movement pattern data sequence when the degree of similarity is highest.

2. The behavior recognition apparatus according to claim 1, further comprising an arm posture estimation unit configured to estimate an arm posture of the user relative to a body of the user, from the hand movement data outputted from the hand movement observation unit,

wherein, when the arm posture matches a predetermined arm posture and the velocity variation detection unit detects the time at which the variation in the velocity of the hand of the user occurs, the feature selection unit is configured to select the hand movement data of the specific times including the detected time at which the variation in the velocity of the hand of the user occurs and the times before and after the detected time.

3. The behavior recognition apparatus according to claim 2, further comprising

a body posture estimation unit configured to estimate a body posture of the user,

wherein, when:

the arm posture matches the predetermined arm posture;

the body posture matches a predetermined body posture; and

the velocity variation detection unit detects the time at which the variation in the velocity of the hand of the user occurs, the feature selection unit is configured to select the hand movement data of the specific times including the detected time at which the variation in the velocity of the hand of the user occurs and the times before and after the detected time.

4. The behavior recognition apparatus according to claim 2,

wherein the standard behavior pattern storage unit is configured to store, in association with the user behavior, a pair of the movement pattern data sequence represented by the hand movement data sequence and the arm posture of the time at which a hand velocity variation occurs, and

the behavior recognition unit is configured to calculate a degree of similarity between:

the pair; and

the hand movement data selected by the feature selection unit and the arm posture estimated by the arm posture estimation unit at the time when the variation in the velocity of the hand of the user occurs, and to recognize, as the user behavior, a behavior represented by the pair when the degree of similarity is highest.

5. The behavior recognition apparatus according to claim 1, further comprising a grip detection unit configured to estimate a grip state when a hand of the user grips an object,

wherein, when the grip state matches a predetermined grip state and the velocity variation detection unit detects the time at which the variation in the velocity of the hand of the user occurs, the feature selection unit is configured to select the hand movement data of the specific times including the detected time at which the variation in the velocity of the hand of the user occurs and the times before and after the detected time.

6. The behavior recognition apparatus according to claim 5,

wherein the standard behavior pattern storage unit is configured to store, in association with the user behavior, a pair of the movement pattern data sequence represented by the hand movement data sequence and the grip state of the time at which the variation in the velocity of the hand of the user occurs, and

the behavior recognition unit is configured to calculate a degree of similarity between:

the pair; and

the hand movement data selected by the feature selection unit and the grip state detected by the grip state detection unit at the time when the variation in the velocity of the hand of the user occurs, and to recognize, as the user behavior, a behavior represented by the pair when the degree of similarity is highest.

7. The behavior recognition apparatus according to claim 2, further comprising a grip detection unit configured to estimate a grip state when a hand of the user grips an object,

wherein, when:

the arm posture matches the predetermined arm posture;

the grip state matches a predetermined grip state; and

the velocity variation detection unit detects the time at which the variation in the velocity of the hand of the user occurs, the feature selection unit is configured to select the hand movement data of the specific times including the detected time at which the variation in the velocity of the hand of the user occurs and the times before and after the detected time.

8. The behavior recognition apparatus according to claim 4, further comprising a grip detection unit configured to estimate a grip state when a hand of the user grips an object,

wherein the standard behavior pattern storage unit is configured to store, in association with the user behavior, a combination of the movement pattern data sequence represented by the hand movement data sequence, the arm posture of the time at which the variation in the velocity of the hand of the user occurs, and the grip state of the time at which the variation in the velocity of the hand of the user occurs, and

the behavior recognition unit is configured to calculate a degree of similarity between:

the combination; and

the hand movement data selected by the feature selection unit, the arm posture estimated by the arm posture estimation unit at the time when the variation in the velocity of the hand of the user occurs, and the grip state detected by the grip detection unit at the time when the variation in the velocity of the hand of the user occurs, and to recognize, as the user behavior, a behavior represented by the combination when the degree of similarity is highest.

9. The behavior recognition apparatus according to claim 1,

wherein the velocity variation detection unit is configured to detect, from the hand movement data, a time at which a hand velocity is equal to or smaller than a predetermined threshold, as the time at which the variation in the velocity of the hand of the user occurs.

10. The behavior recognition apparatus according to claim 1,

wherein, when a cosine distance between a hand velocity vector at a given detection time and a hand velocity vector at a next detection time following the given detection time is below a predetermined threshold, the velocity variation detection unit is configured to detect, from the hand movement data, the given detection time as the time at which the variation in the velocity of the hand of the user occurs.

11. The behavior recognition apparatus according to claim 1,

wherein, when a velocity gradient difference between a hand velocity vector at a given detection time and a hand velocity vector at a next detection time following the given detection time is below a predetermined threshold, the velocity variation detection unit is configured to detect, from the hand movement data, the given detection time as the time at which the variation in the velocity of the hand of the user occurs.

12. A behavior recognition method of recognizing a user behavior from a hand movement made by a user, the behavior recognition method comprising:

detecting the hand movement made by the user and outputting hand movement data indicating the hand movement;

detecting a time at which a variation in velocity of a hand of the user occurs, from the hand movement data outputted in the detecting of a hand movement;

selecting the hand movement data that is outputted in the detecting of a hand movement at specific times including the time detected in the detecting of a time and times before and after the detected time;

a standard behavior pattern storage unit configured to store a movement pattern data sequence; and

calculating a degree of similarity between a movement pattern data sequence stored in a standard behavior pattern storage unit and the hand movement data selected, and recognizing, as the user behavior, a behavior represented by the movement pattern data sequence when the degree of similarity is highest, the movement pattern data sequence being represented by a hand movement data sequence and stored in association with the user behavior.

13. A non-transitory computer-readable recording medium storing a program for recognizing a user behavior from a hand movement made by a user for use in a computer and causing the computer to execute step comprising:

detecting the hand movement made by the user and outputting hand movement data indicating the hand movement;

detecting a time at which a variation in velocity of a hand of the user occurs, from the hand movement data outputted in the detecting of a hand movement;

selecting the hand movement data that is outputted in the detecting of a hand movement at specific times including the time detected in the detecting of a time and times before and after the detected time;

a standard behavior pattern storage unit configured to store a movement pattern data sequence; and

calculating a degree of similarity between a movement pattern data sequence stored in a standard behavior pattern storage unit and the hand movement data selected, and recognizing, as the user behavior, a behavior represented by the movement pattern data sequence when the degree of similarity is highest, the movement pattern data sequence being represented by a hand movement data sequence and stored in association with the user behavior.