3-DIMENSIONAL SHAPE RECONSTRUCTION DEVICE USING DEPTH IMAGE AND COLOR IMAGE AND THE METHOD

• US 20140111507A1
• Filed: 10/16/2013
• Published: 04/24/2014
• Est. Priority Date: 10/23/2012
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: CHOI, Jin Sung, KOO, Bon Ki, KIM, Ho Won
• Granted Patent: US 9,384,585 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation

Abstract

A 3-dimensional (3D) whole shape reconstruction apparatus using a depth image and a color image and a 3D whole shape reconstruction method are provided. The 3D whole shape reconstruction apparatus may include an information generation unit to generate shape information of 3D portions using a depth image and a color image, and a shape reconstruction unit to reconstruct a 3D whole shape of a user by matching the shape information of 3D portions.

Claims

1. A 3-dimensional (3D) whole shape reconstruction apparatus comprising:

an information generation unit to generate shape information of 3D portions using a depth image and a color image; and

a shape reconstruction unit to reconstruct a 3D whole shape of a user by matching the shape information of 3D portions.

2. The 3D whole shape reconstruction apparatus of claim 1, wherein the information generation unit generates the shape information of 3D portions, based on inner silhouette information of the user using the depth image and outer silhouette information of the user using the color image.

3. The 3D whole shape reconstruction apparatus of claim 1, wherein the information generation unit generates the shape information of 3D portions considering characteristics of the depth information and characteristics of the silhouette information of the user, provided by the color image, using a background image.

4. The 3D whole shape reconstruction apparatus of claim 1, wherein the information generation unit extracts a particular pixel of the color image not including depth information of the depth image, and estimates depth information corresponding to the particular pixel.

5. The 3D whole shape reconstruction apparatus of claim 1, wherein the information generation unit generates a color edge image and a color boundary image of the user based on the color image.

6. The 3D whole shape reconstruction apparatus of claim 1, wherein the information generation unit generates an alpha mask image with respect to a foreground image including the user, using difference between the color image and a background image.

7. The 3D whole shape reconstruction apparatus of claim 6, wherein the information generation unit generates depth information of the depth image corresponding to a hole of the alpha mask image, by combining the foreground image with the depth image projected to the color image.

8. The 3D whole shape reconstruction apparatus of claim 1, wherein the shape reconstruction unit reconstructs the 3D whole shape using shape patch information generated based on the shape information of 3D portions.

9. The 3D whole shape reconstruction apparatus of claim 8, wherein the shape patch information is generated by converting a local coordinate expressing the shape information of 3D portions into a global coordinate.

10. The 3D whole shape reconstruction apparatus of claim 8, wherein the shape reconstruction unit reconstructs the 3D whole shape by elliptically modeling the shape patch information and matching the shape patch information with respect to an elliptical model of particular shape patch information according to a movement of the user.

11. A 3-dimensional (3D) whole shape reconstruction method comprising:

generating shape information of 3D portions using a depth image and a color image; and

reconstructing a 3D whole shape of a user by matching the shape information of 3D portions.

12. The 3D whole shape reconstruction method of claim 11, wherein the generating comprises generating the shape information of 3D portions, based on inner silhouette information of the user using the depth image and outer silhouette information of the user using the color image.

13. The 3D whole shape reconstruction method of claim 11, wherein the generating comprises generating the shape information of 3D portions considering characteristics of the depth information and characteristics of the silhouette information of the user, provided by the color image, using a background image.

14. The 3D whole shape reconstruction method of claim 11, wherein the generating comprises extracting a particular pixel of the color image not including depth information of the depth image, and estimating depth information corresponding to the particular pixel.

15. The 3D whole shape reconstruction method of claim 11, wherein the generating comprises generating a color edge image and a color boundary image of the user based on the color image.

16. The 3D whole shape reconstruction method of claim 11, wherein the generating comprises generating an alpha mask image with respect to a foreground image including the user, using difference between the color image and a background image.

17. The 3D whole shape reconstruction method of claim 16, wherein the generating comprises generating depth information of the depth image corresponding to a hole of the alpha mask image, by combining the foreground image with the depth image projected to the color image.

18. The 3D whole shape reconstruction method of claim 11, wherein the reconstructing comprises reconstructing the 3D whole shape using shape patch information generated based on the shape information of 3D portions.

19. The 3D whole shape reconstruction method of claim 18, wherein the shape patch information is generated by converting a local coordinate expressing the shape information of 3D portions into a global coordinate.

20. The 3D whole shape reconstruction method of claim 18, wherein the reconstructing comprises reconstructing

the 3D whole shape by elliptically modeling the shape patch information and matching the shape patch information with respect to an elliptical model of particular shape patch information according to a movement of the user.