APPARATUS AND METHOD FOR ESTIMATING JOINT STRUCTURE OF HUMAN BODY

• US 20130195330A1
• Filed: 01/29/2013
• Published: 08/01/2013
• Est. Priority Date: 01/31/2012
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: KIM, Ho-Won, LEE, Seung-Wook, LEE, Ji-Hyung, KOO, Bon-Ki
• Granted Patent: US 9,058,514 B2
• Time in Patent Office: Days
• US Class Current: 382/128
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/20044 Skeletonization; Medial axis transform
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

Disclosed herein is an apparatus and method for estimating the joint structure of a human body. The apparatus includes a multi-view image acquisition unit for receiving multi-view images acquired by capturing a human body. A human body foreground separation unit extracts a foreground region corresponding to the human body from the acquired multi-view images. A human body shape restoration unit restores voxels indicating geometric space occupation information of the human body using the foreground region corresponding to the human body, thus generating voxel-based three-dimensional (3D) shape information of the human body. A skeleton information extraction unit generates 3D skeleton information from the generated voxel-based 3D shape information of the human body. A skeletal structure estimation unit estimates positions of respective joints from a skeletal structure of the human body using both the generated 3D skeleton information and anthropometric information.

Claims

1. An apparatus for estimating a joint structure of a human body, comprising

a multi-view image acquisition unit for receiving multi-view images acquired by capturing a human body;

a human body foreground separation unit for extracting a foreground region corresponding to the human body from the acquired multi-view images;

a human body shape restoration unit for restoring voxels indicating geometric space occupation information of the human body using the foreground region corresponding to the human body, thus generating voxel-based three-dimensional (3D) shape information of the human body;

a skeleton information extraction unit for generating 3D skeleton information from the generated voxel-based 3D shape information of the human body; and

a skeletal structure estimation unit for estimating positions of respective joints from a skeletal structure of the human body using both the generated 3D skeleton information and anthropometric information.

2. The apparatus of claim 1, wherein the skeleton information extraction unit comprises a medial surface voxel extraction unit for extracting medial surface voxels, which are a minimum number of surface voxels enabling representation of a 3D shape of the human body, from surface voxels in the 3D shape information of the human body.

3. The apparatus of claim 2, wherein the skeleton information extraction unit further comprises a weight calculation unit for calculating weights indicative of distances from a center of gravity of the human body to the respective extracted medial surface voxels.

4. The apparatus of claim 3, wherein the skeleton information extraction unit further comprises a node estimation unit for extracting nodes, which are medial surface voxels corresponding to local minima, from the medial surface voxels based on the weights.

5. The apparatus of claim 4, wherein the skeleton information extraction unit further comprises a root node determination unit for determining a root node corresponding to a root joint in a hierarchical skeletal structure of the human body from the nodes.

6. The apparatus of claim 5, wherein the skeleton information extraction unit further comprises a 3D skeletal structure extraction unit for connecting the nodes along a shortest path based on Dijkstra'"'"'s algorithm by using the root node as a starting point, and then extracting information about 3D skeletons for respective regions of the human body.

7. The apparatus of claim 6, wherein the multi-view image acquisition unit performs camera geometric calibration on each of the multi-view images, input from multiple cameras that capture the human body, based on a global coordinate system located at a center of gravity of the multiple cameras.

8. The apparatus of claim 7, wherein the human body shape restoration unit projects individual voxels in a voxel space, which is constructed based on information about the camera geometric calibration performed by the multi-view image acquisition unit, on each of the multi-view images, thus generating the voxel-based 3D shape information of the human body.

9. The apparatus of claim 8, further comprising a storage unit for storing the information about the camera geometric calibration, the anthropometric information, and information about the hierarchical skeletal structure of the human body.

10. A method of estimating a joint structure of a human body, comprising:

receiving multi-view images acquired by capturing a human body;

extracting a foreground region corresponding to the human body from the acquired multi-view images;

restoring voxels indicating geometric space occupation information of the human body using the foreground region corresponding to the human body, thus generating voxel-based three-dimensional (3D) shape information of the human body;

generating 3D skeleton information from the generated voxel-based 3D shape information of the human body; and

estimating positions of respective joints from a skeletal structure of the human body using both the generated 3D skeleton information and anthropometric information.

11. The method of claim 10, wherein the generating the 3D skeleton information comprises extracting medial surface voxels, which are a minimum number of surface voxels enabling representation of a 3D shape of the human body, from surface voxels in the 3D shape information of the human body.

12. The method of claim 11, wherein the generating the 3D skeleton information further comprises calculating weights indicative of distances from a center of gravity of the human body to the respective medial surface voxels.

13. The method of claim 12, wherein the generating the 3D skeleton information further comprises extracting nodes, which are medial surface voxels corresponding to local minima, from the medial surface voxels based on the weights.

14. The method of claim 13, wherein the generating the 3D skeleton information further comprises determining a root node corresponding to a root joint in a hierarchical skeletal structure of the human body from the nodes.

15. The method of claim 14, wherein the generating the 3D skeleton information further comprises connecting the nodes along a shortest path based on Dijkstra'"'"'s algorithm by using the root node as a starting point, and then extracting information about 3D skeletons for respective regions of the human body.

16. The method of claim 10, wherein the receiving the multi-view images acquired by capturing the human body comprises performing camera geometric calibration on each of the multi-view images, input from multiple cameras that capture the human body, based on a global coordinate system located at a center of gravity of the multiple cameras.

17. The method of claim 16, wherein the voxel-based 3D shape information of the human body is generated by projecting individual voxels in a voxel space, which is constructed based on information about the camera geometric calibration, on each of the multi-view images.