Gesture recognition device and control method for the same

• US 9,349,039 B2
• Filed: 10/20/2014
• Issued: 05/24/2016
• Est. Priority Date: 11/13/2013
• Status: Active Grant

• Current Assignee: Omron Corporation
• Original Assignee: Omron Corporation
• Inventors: Hanzawa, Yuki, Ohta, Takashi, Urabe, Kazuya, Sakai, Shun
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US14/518,570
• Publication Number:
  US 20150131855A1
• Time in Patent Office: 582 Days
• Field of Search: 382/103
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A gesture recognition device configured to detect a gesture from acquired image and generate command issued to a control target instrument according to the gesture, the gesture recognition device comprising: an image acquisition unit configured to acquire an image; a gesture acquisition unit configured to detect a target region performing a gesture from the acquired image, and acquire the gesture based on motion or a shape of the detected target region; a face detection unit configured to detect a face comprised in the acquired image; a correlation unit configured to correlate the detected target region with the detected face using a human body model representing a shape of a human body; a personal identification unit configured to identify a user corresponding to the detected face; and a command generation unit configured to generate a command issued to the control target instrument based on the identified user and the acquired gesture.

Claims

1. A gesture recognition device configured to detect a gesture from an acquired image and generate a command issued to a control target instrument according to the gesture, the gesture recognition device being controlled by a processing unit or control target instrument to operate as:

an image acquisition unit configured to acquire an image;

a gesture acquisition unit configured to detect a target region performing a gesture from the acquired image and acquire the gesture based on motion or a shape of the detected target region;

a face detection unit configured to detect a face comprised in the acquired image;

a correlation unit configured to correlate the detected target region with the detected face using a human body model representing a shape of a human body;

a personal identification unit configured to identify a user corresponding to the detected face;

a command generation unit configured to generate a command issued to the control target instrument based on the identified user and the acquired gesture; and

a gesture storage unit configured to store personal setting data, a set of a gesture and a command issued to the control target instrument being correlated with each user in the personal setting data;

wherein;

the command generation unit is configured to generate the command issued to the control target instrument based on the personal setting data;

the correlation unit is configured to calculate a range where the target region is estimated to exist for each detected face using a coordinate of the detected face and the human body model and correlate the target region and the face with each other when the target region exists in the calculated range; and

the personal identification unit is configured to identify the user through acquiring a user identifier corresponding to the face correlated with the target region by comparing feature amount of the correlated face with the personal setting data.

2. The gesture recognition device according to claim 1, wherein:

the gesture storage unit is further configured to store priority correlated with a user therein, and

the command generation unit is configured to process a gesture corresponding to a user having higher priority when the gesture acquisition unit simultaneously acquires at least two gestures.

3. The gesture recognition device according to claim 1, wherein the command generation unit is configured to generate a default command corresponding to a gesture without considering a user performing the gesture when the correlation unit fails to correlate the detected target region and the detected face with each other.

4. The gesture recognition device according to claim 1, wherein the gesture recognition device being controlled by the processing unit or control target instrument is configured to operate as:

a user estimation unit configured to estimate which user performs the gesture without use of a face detection result, wherein the user estimation unit is configured to estimate the user performing the gesture when the correlation unit fails to correlate the detected target region and the detected face with each other.

5. A method for controlling a gesture recognition device configured to detect a gesture from an acquired image and generate a command issued to a control target instrument according to the gesture, the method comprising:

acquiring an image;

detecting a target region performing a gesture from the acquired image and acquiring the gesture based on motion or a shape of the detected target region;

detecting a face comprised in the acquired image;

correlating the detected target region with the detected face by applying a human body model to the acquired image;

identifying a user corresponding to the detected face;

a command generation step for generating a command issued to the control target instrument based on the identified user and the acquired gesture;

storing personal setting data, a set of a gesture and a command issued to the control target instrument being correlated with each user in the personal setting data; and

wherein the method further includes;

generating the command issued to the control target instrument based on the personal setting data;

calculating a range where the target region is estimated to exist for each detected face using a coordinate of the detected face and the human body model;

correlating the target region and the face with each other when the target region exists in the calculated range; and

identifying the user through acquiring a user identifier corresponding to the face correlated with the target region by comparing feature amount of the correlated face with the personal setting data.

6. A non-transitory computer readable medium having stored thereon a gesture recognition program configured to cause a gesture recognition device, configured to detect a gesture from an acquired image and generate a command issued to a control target instrument according to the gesture, to:

acquire an image;

detect a target region performing a gesture from the acquired image and the gesture based on motion or a shape of the detected target region;

detect a face comprised in the acquired image;

correlate the detected target region with the detected face by applying a human body model to the acquired image;

identify a user corresponding to the detected face;

generate a command issued to the control target instrument based on the identified user and the acquired gesture; and

store personal setting data, a set of a gesture and a command issued to the control target instrument being correlated with each user in the personal setting data;

wherein the gesture recognition program configured to cause a gesture recognition device to;

generate the command issued to the control target instrument based on the personal setting data;

calculate a range where the target region is estimated to exist for each detected face using a coordinate of the detected face and the human body model;

correlate the target region and the face with each other when the target region exists in the calculated range; and

identify the user through acquiring a user identifier corresponding to the face correlated with the target region by comparing feature amount of the correlated face with the personal setting data.