Gesture recognition apparatus, method for controlling gesture recognition apparatus, and control program

• US 8,526,675 B2
• Filed: 02/25/2011
• Issued: 09/03/2013
• Est. Priority Date: 03/15/2010
• Status: Active Grant

• Current Assignee: Omron Corporation
• Original Assignee: Omron Corporation
• Inventors: Ruan, Xiang
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US13/035,107
• Publication Number:
  US 20110222726A1
• Time in Patent Office: 921 Days
• Field of Search: 382/103, 382/107, 382/236, 348169-172, 348/352, 715/863
• US Class Current: 382/103
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A gesture recognition apparatus is caused to correctly recognize start and end of a gesture without use of special unit by a natural manipulation of a user and low-load processing for the gesture recognition apparatus. The gesture recognition apparatus that recognizes the gesture from action of a recognition object taken in a moving image includes: a gravity center tracking unit that detects a specific subject having a specific feature from the moving image; a moving speed determining unit that computes a moving speed per unit time of the specific subject; a moving pattern extracting unit that extracts a moving pattern of the specific subject; and a start/end judgment unit that discriminates movement of the specific subject as an instruction (such as an instruction to start or end gesture recognition processing) input to the gesture recognition apparatus when the moving speed and the moving pattern satisfy predetermined conditions.

Claims

1. A gesture recognition apparatus that performs gesture recognition processing of recognizing a gesture from action of a recognition object taken in a moving image, the gesture recognition apparatus comprising:

a gesture recognizing unit for performing the gesture recognition processing;

a specific subject detecting unit for detecting a specific subject having a specific feature from the moving image;

a moving speed computing unit for computing a moving speed per unit time of the specific subject;

a moving pattern extracting unit for extracting a moving pattern of the specific subject; and

a start/end judgment unit for judging movement of the specific subject as an instruction of start or end of the gesture recognition processing when the moving speed and the moving pattern satisfy predetermined conditions,

wherein, if the start/end judgment unit starts the gesture recognition processing, the gesture recognizing unit performs the gesture recognition processing by recognizing a shape or a posture of the specific subject.

2. The gesture recognition apparatus according to claim 1, wherein the specific subject detecting unit detects an organ of a user or part of the organ as the specific subject.

3. The gesture recognition apparatus according to claim 2, wherein the specific subject detecting unit detects a palm of the user as the specific subject.

4. The gesture recognition apparatus according to claim 3, further comprising:

a gesture recognition unit for performing the gesture recognition processing by discriminating a finger posture of the user as the recognition object taken in a moving image,

wherein the shape or the posture of the specific subject is a finger posture of the user;

wherein the gesture recognition unit discriminates the finger posture from a frame of the moving image from a start point to an end point, and

wherein the start point and the end point are judged by the start/end judgment unit.

5. The gesture recognition apparatus according to claim 1, wherein the specific subject detecting unit detects a user'"'"'s possession having a specific feature in color or shape as the specific subject.

6. The gesture recognition apparatus according to claim 1, wherein the input instruction discriminating unit is a gesture recognition unit for recognizing the movement of the specific subject as the gesture based on the moving speed and the moving pattern of the specific subject.

7. A gesture recognition apparatus that performs gesture recognition processing of recognizing a gesture from action of a recognition object taken in a moving image, the gesture recognition apparatus comprising:

a specific subject detecting unit for detecting a specific subject having a specific feature from the moving image;

a moving speed computing unit for computing a moving speed per unit time of the specific subject;

a moving pattern extracting unit for extracting a moving pattern of the specific subject; and

an input instruction discriminating unit for discriminating movement of the specific subject as an instruction input to the gesture recognition apparatus when the moving speed and the moving pattern satisfy predetermined conditions,

wherein the specific subject detecting unit specifies a specific subject region including at least part of the specific subject on a frame of the moving image and specifies a gravity center position coordinate in the specific subject region.

8. The gesture recognition apparatus according to claim 7, wherein the moving speed computing unit normalizes a moving distance per unit time of the gravity center in the specific subject region by a size of the specific subject region and computes the moving speed based on a gravity center moving amount obtained by the normalization.

9. The gesture recognition apparatus according to claim 7, wherein the moving pattern extracting unit extracts a variance of the gravity center position coordinates with respect to the gravity centers of the specific subject regions, the gravity centers of the specific subject regions being specified with respect to the plurality of frames in the moving image.

10. The gesture recognition apparatus according to claim 9, wherein the input instruction discriminating unit discriminates the movement of the specific subject as the instruction of the start or end of the gesture recognition processing, when the moving speed computed by the moving speed computing unit is more than a predetermined threshold, and when the variance of the gravity center position coordinate extracted by the moving pattern extracting unit falls within a range from a predetermined lower threshold to a predetermined upper threshold.

11. A method for controlling a gesture recognition apparatus that performs gesture recognition processing of recognizing a gesture from action of a recognition object taken in a moving image, the gesture recognition apparatus controlling method comprising the steps of:

performing the gesture recognition processing;

detecting a specific subject having a specific feature from the moving image;

computing a moving speed per unit time of the specific subject;

extracting a moving pattern of the specific subject; and

judging movement of the specific subject as an instruction of start or end of the gesture recognition processing when the moving speed and the moving pattern satisfy predetermined conditions,

wherein, if the gesture recognition processing is started based on the judgment of the movement of the specific subject, the gesture recognizing unit performs the gesture recognition processing is performed by recognizing a shape or a posture of the specific subject.

12. A gesture recognition apparatus that performs gesture recognition processing of recognizing a gesture from action of a recognition object taken in a moving image, the gesture recognition apparatus comprising:

a specific subject detecting unit for detecting a specific subject having a specific feature from the moving image;

a moving speed computing unit for computing a moving speed per unit time of the specific subject;

a moving pattern extracting unit for extracting a moving pattern of the specific subject; and

an input instruction discriminating unit for discriminating movement of the specific subject as an instruction input to the gesture recognition apparatus when the moving speed and the moving pattern satisfy predetermined conditions,

wherein the input instruction discriminating unit comprises a start/end judgment unit for judging the action of the specific subject as an instruction of start or end of the gesture recognition processing based on the moving speed and the moving pattern of the specific subject; and

,

wherein the specific subject detecting unit specifies a specific subject region including at least part of the specific subject on a frame of the moving image and specifies a gravity center position coordinate in the specific subject region.

13. The gesture recognition apparatus according to claim 12, wherein the moving speed computing unit normalizes a moving distance per unit time of the gravity center in the specific subject region by a size of the specific subject region and computes the moving speed based on a gravity center moving amount obtained by the normalization.

14. The gesture recognition apparatus according to claim 12, wherein the moving pattern extracting unit extracts a variance of the gravity center position coordinates with respect to the gravity centers of the specific subject regions, the gravity centers of the specific subject regions being specified with respect to the plurality of frames in the moving image.

15. The gesture recognition apparatus according to claim 12, wherein the input instruction discriminating unit discriminates the movement of the specific subject as the instruction of the start or end of the gesture recognition processing, when the moving speed computed by the moving speed computing unit is more than a predetermined threshold, and when the variance of the gravity center position coordinate extracted by the moving pattern extracting unit falls within a range from a predetermined lower threshold to a predetermined upper threshold.

16. A gesture recognition apparatus that performs gesture recognition processing of recognizing a gesture from action of a recognition object taken in a moving image, the gesture recognition apparatus comprising:

a specific subject detecting unit for detecting a specific subject having a specific feature from the moving image;

a moving speed computing unit for computing a moving speed per unit time of the specific subject;

a moving pattern extracting unit for extracting a moving pattern of the specific subject;

an input instruction discriminating unit for discriminating movement of the specific subject as an instruction input to the gesture recognition apparatus when the moving speed and the moving pattern satisfy predetermined conditions; and

a gesture recognition unit for performing the gesture recognition processing by discriminating a finger posture of the user as the recognition object taken in a moving image,

wherein the input instruction discriminating unit comprises a start/end judgment unit for judging the action of the specific subject as an instruction of start or end of the gesture recognition processing based on the moving speed and the moving pattern of the specific subject;

wherein the specific subject detecting unit detects an organ of a user or part of the organ as the specific subject;

wherein the specific subject detecting unit detects a palm of the user as the specific subject;

wherein the gesture recognition unit discriminates the finger posture from a frame of the moving image from a start point to an end point;

wherein the start point and the end point are judged by the start/end judgment unit; and

wherein the specific subject detecting unit specifies a specific subject region including at least part of the specific subject on a frame of the moving image and specifies a gravity center position coordinate in the specific subject region.

17. The gesture recognition apparatus according to claim 16, wherein the moving speed computing unit normalizes a moving distance per unit time of the gravity center in the specific subject region by a size of the specific subject region and computes the moving speed based on a gravity center moving amount obtained by the normalization.

18. The gesture recognition apparatus according to claim 16, wherein the moving pattern extracting unit extracts a variance of the gravity center position coordinates with respect to the gravity centers of the specific subject regions, the gravity centers of the specific subject regions being specified with respect to the plurality of frames in the moving image.

19. A non-transitory computer readable medium for controlling a gesture recognition apparatus that performs gesture recognition processing of recognizing a gesture from action of a recognition object taken in a moving image, the gesture recognition apparatus controlling method comprising the steps of:

performing the gesture recognition processing;

detecting a specific subject having a specific feature from the moving image;

computing a moving speed per unit time of the specific subject;

extracting a moving pattern of the specific subject; and

judging movement of the specific subject as an instruction of start or end of the gesture recognition processing when the moving speed and the moving pattern satisfy predetermined conditions,

wherein, if the gesture recognition processing is started based on the judgment of the movement of the specific subject, the gesture recognizing unit performs the gesture recognition processing is performed by recognizing a shape or a posture of the specific subject.