Gesture recognition apparatus and method thereof

• US 8,405,712 B2
• Filed: 09/19/2008
• Issued: 03/26/2013
• Est. Priority Date: 09/25/2007
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation), Toshiba Digital Solutions Corporation (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Ike, Tsukasa, Okamoto, Yasukazu, Ishigaki, Satoru, Sakamoto, Kei, Fujita, Masahiro, Kazama, Hisashi
• Primary Examiner(s): SHAW, PELING ANDREW
• Application Number:
  US12/233,693
• Publication Number:
  US 20090153655A1
• Time in Patent Office: 1,649 Days
• Field of Search: 715/700, 715/863, 345156-184, 342/28, 342/114, 382/103, 382/107, 382/122, 382/216, 382/236, 348/154, 348/155, 725/12, 352/39, 396/153
• US Class Current: 348/77
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A gesture recognition apparatus including a detector, a controller and a gesture recognizer, and is able to detect a portion to be detected using an image captured at a first resolution without reducing the number of images to be entered per unit of time to an object detector by detecting an auxiliary portion instead when the portion to be detected is not detected at a second resolution, estimating the position of the auxiliary portion based on the position of the auxiliary portion, and extracting the periphery thereof from the image captured at the first resolution.

Claims

1. A gesture recognition apparatus comprising:

a first detector configured to detect a detection area including a portion of a body of an operator in a first image of a motion image captured at a first resolution;

a second detector configured to detect an auxiliary area including an auxiliary portion used for detecting the portion in a second image of the motion image captured at the first resolution when the detection area cannot be detected from the first image of the motion image;

a physical controller configured to control, when the auxiliary area is detected, a capturing unit to capture a third image of the motion image, which is an image in the frame next to the second image and is captured at a second resolution higher than the first resolution, by capturing an area estimated from a predetermined positional relationship between the portion of a body and the auxiliary portion;

a third detector configured to detect the detection area in the third image; and

a gesture recognizer configured to recognize a gesture of the portion based on an entire area of the first image when the detection area is detected from the first image and based on the detection area of the third image when the detection area cannot be detected from the first image.

2. The apparatus according to claim 1, wherein the second image used in the second detector is an image in a frame next to the first image, or the same image as the first image.

3. The apparatus according to claim 1, wherein the third image used in the third detector is an image in a frame next to the second image.

4. The apparatus according to claim 1, wherein the auxiliary area detected by the second detector is an area having a larger size than the detection area.

5. The apparatus according to claim 1, wherein the portion to be detected is the hand of the operator and the auxiliary area is the face of the operator.

6. The apparatus according to claim 1, further comprising an image input unit which is configured to change the resolution for each or every some frames when inputting the motion image.

9. A gesture recognition apparatus according to claim 1, wherein the controller is configured to detect the detection area from an image in a frame next to the second image by the first detector when the second detector cannot detect the auxiliary area.

10. A gesture recognition apparatus according to claim 1, wherein the controller is configured to detect the detection area from an image in a frame next to the second image by the first detector when the size of the auxiliary area is larger than the threshold value.

11. A gesture recognition apparatus according to claim 1, wherein the controller is configured to control, when the size of the auxiliary area is larger than the threshold value, a capturing unit to capture a third image of the motion image, which is an image in the frame next to the second image and is captured at a second resolution higher than the first resolution, based on an area estimated from a predetermined positional relationship between the portion of a body and the auxiliary portion.

7. A computer-implemented method of recognizing gestures comprising:

detecting a detection area including a portion of a body of an operator in a first image of a motion image captured at a first resolution;

detecting an auxiliary area including an auxiliary portion used for detecting the portion in a second image of the motion image captured at the first resolution when the detection area cannot be detected from the first image of the motion image;

controlling, when the auxiliary area is detected and using a physical controller, a capturing unit to capture a third image of the motion image, which is an image in the frame next to the second image and is captured at a second resolution higher than the first resolution, by capturing an area estimated from a predetermined positional relationship between the portion of the body and the auxiliary portion;

detecting the detection area in the third image; and

recognizing a gesture of the portion based on an entire area of the first image when the detection area is detected from the first image and based on the detection area of the third image when the detection area cannot be detected from the first image.

8. A non-transitory computer-readable medium storing a set of instructions that when executed by processor perform a method, the method comprising:

detecting a detection area including a portion of a body of an operator in a first image of a motion image captured at a first resolution;

detecting an auxiliary area including an auxiliary portion used for detecting the portion in a second image of the motion image captured at the first resolution when the detection area cannot be detected from the first image of the motion image;

controlling, when the auxiliary area is detected and using a physical controller, a capturing unit to capture a third image of the motion image, which is an image in the frame next to the second image and is captured at a second resolution higher than the first resolution by capturing an area estimated from a predetermined positional relationship between the portion of the a body and the auxiliary portion;

detecting the detection area in the third image; and

recognizing a gesture of the portion based on an entire area of the first image when the detection area is detected from the first image and based on the detection area of the third image when the detection area cannot be detected from the first image.