Method and apparatus for generating three-dimensional model information

• US 8,055,061 B2
• Filed: 02/22/2008
• Issued: 11/08/2011
• Est. Priority Date: 02/27/2007
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Katano, Yasuo
• Primary Examiner(s): Carter; Aaron W
• Application Number:
  US12/035,522
• Publication Number:
  US 20080204453A1
• Time in Patent Office: 1,355 Days
• Field of Search: 382/154, 382/103, 382/284, 382/285, 345629-633
• US Class Current: 382/154
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects

Abstract

A method forms a region image extracting a region of a physical object which generates three-dimensional model information from a video image of physical space. Then, from position and orientation information for the physical object and from the region image, a primitive virtual object of a size to encompass the region image is generated. From the primitive virtual object a virtual object having a shape according to the region image is generated, and the three-dimensional model information is generated as three-dimensional model information representing the physical object.

Claims

1. A method for generating a virtual space object which has a similar shape as a predetermined physical space object, comprising:

an input step of inputting a captured image in which said physical space object is captured;

an extraction step of extracting a region of said physical space object from said captured image;

a generation step of generating an initial virtual space object having a position and orientation corresponding to a position and orientation of said physical space object, said virtual space object having a size to encompass said extracted region;

a cutoff step of generating a cut off virtual space object by cutting off part of the initial virtual space object existing at positions not corresponding to said extracted region from a viewpoint from which said captured image is captured;

a composition step of generating a composite image of said captured image and a virtual image of the cut off virtual space object, and

an output step of outputting said composite image.

2. The method according to claim 1, wherein said generation step generates said initial virtual space object having no thickness.

3. The method according to claim 2, further comprising a thickness provision step of providing thickness to the cut off virtual space object;

and

wherein said composition step of generating a composite image of said captured image and a virtual image of the cut off virtual space object which is provided the thickness.

4. The method according to claim 1, wherein said generation step generates said initial virtual space object having a thickness according to the orientation of said physical space object.

6. A non-transitory computer-readable recording medium on which is recorded a program for causing a computer to execute the method according to claim 1.

7. The method according to claim 1, wherein the virtual object moves following to the physical object, and

the method further comprising:

a repeat input step of inputting another captured image including said physical space object which has moved in the position and/or orientation from the position and/or orientation at which the virtual space object was captured said captured image on the input step;

a repeat extraction step of extracting a second region of the physical space object which has moved, from the other captured image input in said repeat input step; and

a repeat cutoff step of further cutting off part of the cut off virtual space object existing at positions not corresponding to said second region from a viewpoint from which said the other captured image is captured.

8. The method according to claim 7 further comprising:

a repeat composition step of generating a second composite image of the other captured image which input in said repeat input step and a virtual image of the cut off virtual space object which is further cut off in the repeat cutoff step; and

a repeat output step of outputting said composite image generated in said repeat composition step.

9. The method according to claim 7, further comprising:

a measurement step of measuring a position and orientation of said physical space object;

wherein the virtual space object moves based on said measured position and orientation in order to follow a movement of the physical space object.

10. The method according to claim 9, wherein said measurement step measures said position and orientation based on an output from a sensor which attached on the physical flat object for laying said physical space object on said physical flat object.

5. An apparatus for generating a virtual space object which has a similar shape as a predetermined physical space object, comprising:

an input unit which inputs a captured video image in which said physical space object is captured;

an extraction unit which extracts a region of said physical space object from said captured image;

a generation unit which generates an initial virtual space object having a position and orientation according to a position and orientation of said physical space object, said virtual space object having a size to encompass said extracted region;

a cutoff unit which generates a cut off virtual space object by cutting off part of the initial virtual space object existing at positions not corresponding to said extracted region from a viewpoint from which said captured image is captured; and

an output unit which outputs said composite image.