Apparatus and method for detecting multiple arms and hands by using three-dimensional image

• US 9,286,694 B2
• Filed: 07/23/2014
• Issued: 03/15/2016
• Est. Priority Date: 01/10/2014
• Status: Active Grant

• Current Assignee: Anyractive Co., Ltd.
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Kim, Jin Ho, Jo, Dong Sik, Kim, Hang Kee, Kim, Hye Mi, Kim, Ki Hong, Kim, Dae Hwan
• Primary Examiner(s): Kim, Ahshik
• Application Number:
  US14/338,933
• Publication Number:
  US 20150199824A1
• Time in Patent Office: 601 Days
• Field of Search: 235/462.27, 235/454, 345/156, 345/158, 715/863
• US Class Current: 1/1
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Provided are an apparatus and method for detecting a plurality of arms and hands by using a three-dimensional (3D) image. The apparatus includes an image input unit configured to acquire a 3D image of an object, an arm detecting unit configured to detect one or more component-unit candidate regions of the object in the 3D image, and detect one or more arm regions by using arm detection feature information, extracted from each of the candidate regions, and a pattern recognition algorithm, and a hand detecting unit configured to calculate a position of a hand and a position of a wrist in each of the arm regions detected by the arm detecting unit, and detect a hand region by using the position of the hand and the position of the wrist.

Claims

1. An apparatus for detecting a plurality of arms and hands by using a three-dimensional (3D) image, the apparatus comprising:

an image input unit configured to acquire a 3D image of an object;

an arm detecting unit configured to detect one or more component-unit candidate regions of the object in the 3D image, and detect one or more arm regions by using arm detection feature information, extracted from each of the candidate regions, and a pattern recognition algorithm; and

a hand detecting unit configured to calculate a position of a hand and a position of a wrist in each of the arm regions detected by the arm detecting unit, and detect a hand region by using the position of the hand and the position of the wrist.

2. The apparatus of claim 1, wherein by using a cascade technique, the arm detecting unit sequentially excludes candidate regions, from which feature information having a low correlation with feature information of an arm are extracted, among the detected candidate regions.

3. The apparatus of claim 1, wherein the arm detecting unit comprises:

an object detector configured to detect the one or more component-unit candidate regions of the object in the 3D image;

an arm feature extractor configured to extract the arm detection feature information from each of the candidate regions; and

an arm object determiner configured to, by using a cascade technique, sequentially exclude candidate regions, from which feature information having a low correlation with feature information of an arm are extracted, among the detected candidate regions to determine an arm candidate region.

4. The apparatus of claim 1, wherein the feature information comprises at least one of a texture, a color, a shape, a length, a contour line, and a parallel line.

5. The apparatus of claim 1, wherein the hand detecting unit determines the hand region by using a classifier that has been learned by using the position of the hand and the position of the wrist as feature information.

6. The apparatus of claim 1, wherein the hand detecting unit detects a hand candidate region from each arm region, sets a region having a predetermined volume of a hand in a 3D pixel connection path direction with respect to an uppermost pixel in a long axis direction of an arm in the hand candidate region, and determines, as the position of the hand, an average position coordinate value of pixels included in the set region.

7. The apparatus of claim 1, wherein the hand detecting unit sets a polar coordinate system with the position of the hand as an origin, performs an angle-based log search, and searches for a wrist region while sequentially moving between paths.

8. The apparatus of claim 1, wherein the hand detecting unit determines, as the position of the wrist, a point for which an inflection point is generated on a moving path, in which there is a parallel line with respect to the moving path, among a plurality of paths with the position of the hand as an origin in a 3D pixel direction.

9. An apparatus for detecting a plurality of arms and hands by using a three-dimensional (3D) image, the apparatus comprising:

an image input unit configured to acquire a real-time 3D image of an object;

an arm detecting unit configured to detect one or more component-unit candidate regions of the object in a current 3D image frame, and detect one or more arm regions by using arm detection feature information, extracted from each of the candidate regions, and a pattern recognition algorithm;

an arm tracking unit configured to track an arm by using an association between a first arm track feature of each of one or more arm regions detected from the current 3D image frame and a second arm track feature of each of one or more arm regions detected from a previous 3D image frame;

a hand detecting unit configured to calculate a position of a hand and a position of a wrist in each arm region tracked by the arm tracking unit, and detect a hand region by using the position of the hand and the position of the wrist; and

a hand tracking unit configured to track a hand by using an association between a first hand track feature of each of one or more hand regions detected from the current 3D image frame and a second hand track feature of each of one or more hand regions detected from the previous 3D image frame.

10. The apparatus of claim 9, wherein by using a cascade technique, the arm detecting unit sequentially excludes candidate regions, from which feature information having a low correlation with feature information of an arm are extracted, among the detected candidate regions.

11. The apparatus of claim 9, wherein the arm tracking unit evaluates a similarity between the first arm track feature of each of the one or more arm regions detected from the current 3D image frame and the second arm track feature of each of the one or more arm regions detected from the previous 3D image frame to track an arm.

12. The apparatus of claim 9, wherein the hand detecting unit determines the hand region by using a classifier that has been learned by using the position of the hand and the position of the wrist as feature information.

13. The apparatus of claim 9, wherein the hand detecting unit detects a hand candidate region from each arm region, sets a region having a predetermined volume of a hand in a 3D pixel connection path direction with respect to an uppermost pixel in a long axis direction of an arm in the hand candidate region, and determines, as the position of the hand, an average position coordinate value of pixels included in the set region.

14. The apparatus of claim 9, wherein the hand detecting unit sets a polar coordinate system with the position of the hand as an origin, performs an angle-based log search, and searches for a wrist region while sequentially moving between paths.

15. The apparatus of claim 9, further comprising a hand/arm information storage unit configured to store the first and second arm track features and the first and second hand track features.

16. A method of detecting a plurality of arms and hands by using a three-dimensional (3D) image, the method comprising:

acquiring a real-time 3D image of an object;

detecting one or more component-unit candidate regions of the object in a current 3D image frame, and detecting one or more arm regions by using arm detection feature information, extracted from each of the candidate regions, and a pattern recognition algorithm;

tracking an arm by using an association between a first arm track feature of each of one or more arm regions detected from the current 3D image frame and a second arm track feature of each of one or more arm regions detected from a previous 3D image frame;

calculating a position of a hand and a position of a wrist in each tracked arm region, and detecting a hand region by using the position of the hand and the position of the wrist;

tracking a hand by using an association between a first hand track feature of each of one or more hand regions detected from the current 3D image frame and a second hand track feature of each of one or more hand regions detected from the previous 3D image frame; and

updating a database with the first arm hand track feature and the first hand track feature.