Method for reconstructing three-dimensional information of object and apparatus for the same

• US 10,643,372 B2
• Filed: 12/08/2017
• Issued: 05/05/2020
• Est. Priority Date: 11/22/2017
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Han, Byung Ok, Kim, Jae Hwan, Kim, Ho Won, Jung, Yu Gu, Park, Chang Joon, Lee, Gil Haeng
• Primary Examiner(s): Mariam, Daniel G
• Application Number:
  US15/836,542
• Publication Number:
  US 20190156552A1
• Time in Patent Office: 879 Days
• Field of Search: 382154
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/55 from multiple images

Abstract

The present disclosure provides a method and apparatus for reconstructing 3D information of an object. The method of reconstructing 3D information of an object may include: reconstructing 3D information of a background space, using at least one of images of the background space for three-dimensionally reconstructing the object; separating the object from the background space on the basis of at least one of images of the object in the background space, and of obtaining a depth image of the object separated from the background space; and generating 3D information of the object based on images taken by at least one of cameras in consideration of position relationships between the depth image of the object and the cameras.

Claims

1. A method of reconstructing 3D information of an object, the method comprising:

reconstructing 3D information of a background space, using at least one of images of the background space for three-dimensionally reconstructing the object;

separating the object from the background space on the basis of at least one of images of the object in the background space, and of obtaining a depth image of the object separated from the background space; and

generating 3D information of the object based on images taken by at least one of cameras in consideration of position relationships between the depth image of the object and the cameras,

wherein the generating of the 3D information of the object includes;

setting an object volume space corresponding to an area where the object exists; and

generating a state map of the object volume space, using the 3D information of the background space and the depth image of the object.

2. The method of claim 1, wherein the obtaining of the depth image of the object includes determining the 3D information of the background space and outlier based on correspondence relationship information between objects in the background space.

3. The method of claim 2, wherein the obtaining of the depth image of the object further includes removing noise from the images of the object.

4. The method of claim 1, wherein the generating of the 3D information of the object further includes removing image information corresponding to a substance holding the object on the basis of a state change technique that uses slow adaptation.

5. The method of claim 4, wherein the state change technique using slow adaptation determines elements existing over a predetermined time in the object volume space as elements constituting the object and determines elements existing under the predetermined time in the object volume space as elements constituting the substance holding the object.

6. The method of claim 1, wherein the generating of the 3D information of the object further includes:

setting relative positions between the object volume space and the cameras; and

determining a change in the relative positions between the object volume space and the cameras and generating the 3D information of the object corresponding to the relative positions between the object volume space and the cameras.

7. The method of claim 1, wherein the images of the background space, the images of the object, and the images taken by the cameras are depth images.

8. An apparatus for reconstructing 3D information of an object, the apparatus comprising:

at least one of cameras taking images for reconstructing 3D information;

a background information reconstructor reconstructing 3D information of a background space for three-dimensionally reconstructing an object, using at least one of images of the background space;

a background/object separator separating the object from the background space on the basis of at least one of images of the object in the background space, and obtaining a depth image of the object separated from the background space; and

a 3D information reconstructor generating 3D information of the object based on images taken by the cameras in consideration of positional relationships between the depth image of the object and the cameras,

wherein the 3D information reconstructor sets an object volume space corresponding to an area where the object exists, and generates a state map of the object volume space, using the 3D information of the background space and the depth image of the object.

9. The apparatus of claim 8, wherein the background/object separator determines the 3D information of the background space and outlier based on correspondence relationship information between objects in the background space, and generates a depth image of the object using the outlier.

10. The apparatus of claim 9, wherein the background/object separator includes a noise remover removing noise information from the images of the object.

11. The apparatus of claim 8, wherein the 3D information reconstructor removes image information corresponding to a substance holding the object on the basis of a state change technique that uses slow adaptation.

12. The apparatus of claim 11, wherein the state change technique using slow adaptation determines elements existing over a predetermined time in the object volume space as elements constituting the object and determines elements existing under the predetermined time in the object volume space as elements constituting the substance holding the object.

13. The apparatus of claim 8, wherein the 3D information reconstructor sets relative positions between the object volume space and the cameras, determines a change in the relative positions between the object volume space and the cameras, and generates the 3D information of the object corresponding to the relative positions between the object volume space and the cameras.

14. The apparatus of claim 8, wherein the cameras are depth cameras taking depth images.