IMAGE PROCESSING APPARATUS FOR COMBINING REAL OBJECT AND VIRTUAL OBJECT AND PROCESSING METHOD THEREFOR

• US 20090128552A1
• Filed: 11/05/2008
• Published: 05/21/2009
• Est. Priority Date: 11/07/2007
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Katano, Yasuo, Hashimoto, Takayuki, Fujiki, Masakazu, Tomite, Kaname
• Granted Patent: US 9,082,213 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 15/06 Ray-tracing
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

When an image in a virtual space in which a virtual object is arranged is generated using a ray tracing method, and when it is determined that a ray which is generated in accordance with the ray tracing method successively intersected an approximate virtual object such as a hand which is a real object at lest twice, an image corresponding to a first intersection is generated in accordance with the ray emitted to the first intersection.

Claims

1. An image processing apparatus that generates an image in a virtual space in which a virtual object is arranged using a ray tracing method, comprising:

an arrangement unit configured to arrange an approximate virtual object which includes at least one virtual element and which has a shape obtained by approximating a shape of a real object in the virtual space;

a determination unit configured to determine an intersection of a ray generated in accordance with the ray tracing method and the approximate virtual object; and

a generation unit configured to generate, when it is determined that the ray successively intersected the approximate virtual object at least twice, an image corresponding to a first intersection in accordance with the ray emitted to the first intersection.

2. The image processing apparatus according to claim 1,

wherein after the ray successively intersected the approximate virtual object, when the ray intersected a virtual object or another approximate virtual object, the generation unit generates a pixel in accordance with the ray that intersected the virtual object.

3. The image processing apparatus according to claim 1,

wherein after the ray successively intersected the approximate virtual object, the generation unit obtains a distance between intersections, and when it is determined that the distance is smaller than the virtual object, the generation unit generates a pixel in accordance with the ray emitted to the first intersection.

4. An image processing apparatus that generates an image in a virtual space in which a virtual object is arranged using a ray tracing method, comprising:

an extraction unit configured to extract a region corresponding to a real object included in a photographed image obtained using a camera;

an arrangement unit configured to arrange an approximate virtual object which includes at least one virtual element and which has a shape obtained by approximating a shape of a real object in the virtual space; and

a generation unit configured to generate an image by combining an image of a virtual object different from the approximate virtual object, which is projected in the virtual space with an image of the real object using the ray tracing method,

wherein the generation unit generates an image which is located out of the region extracted using the extraction unit and which is included in a region corresponding to the approximate virtual object by performing an operation of detecting an intersection of a ray and a virtual object using the ray tracing method.

5. An image processing method that generates an image in a virtual space in which a virtual object is arranged using a ray tracing method, comprising:

arranging an approximate virtual object which includes at least one virtual element and which has a shape obtained by approximating a shape of a real object in the virtual space;

determining an intersection of a ray generated in accordance with the ray tracing method and the approximate virtual object; and

generating, when it is determined that the ray successively intersected the approximate virtual object at least twice, an image corresponding to a first intersection in accordance with the ray emitted to the first intersection.

6. An image processing method that generates an image in a virtual space in which a virtual object is arranged using a ray tracing method, comprising:

extracting a region corresponding to a real object included in a photographed image obtained using a camera;

arranging an approximate virtual object which includes at least one virtual element and which has a shape obtained by approximating a shape of a real object in the virtual space; and

generating an image by combining an image of a virtual object different from the approximate virtual object, which is projected in the virtual space with an image of the real object using the ray tracing method,

wherein an image which is located out of the extracted region and which is included in a region corresponding to the approximate virtual object is generated by performing an operation of detecting an intersection of a ray and a virtual object using the ray tracing method.

7. A computer-readable storage medium which stores therein a computer program which causes a computer to execute an image processing method for generating an image in a virtual space in which a virtual object is arranged using a ray tracing method, comprising:

arranging an approximate virtual object which includes at least one virtual element and which has a shape obtained by approximating a shape of a real object in the virtual space;

determining an intersection of a ray generated in accordance with the ray tracing method and the approximate virtual object; and

generating, when it is determined that the ray successively intersected the approximate virtual object at least twice, an image corresponding to a first intersection in accordance with the ray emitted to the first intersection.

8. A computer-readable storage medium which stores therein a computer program which causes a computer to execute an image processing method for generating an image in a virtual space in which a virtual object is arranged by means of a ray tracing method, comprising:

extracting a region corresponding to a real object included in a photographed image obtained using a camera;

arranging an approximate virtual object which includes at least one virtual element and which has a shape obtained by approximating a shape of a real object in the virtual space; and

generating an image by combining an image of a virtual object different from the approximate virtual object, which is projected in the virtual space with an image of the real object in accordance with the ray tracing method,

wherein an image which is located out of the extracted region and which is included in a region corresponding to the approximate virtual object is generated by performing an operation of detecting an intersection of a ray and a virtual object in accordance with the ray tracing method.

9. An image processing apparatus which renders a shadow in a three-dimensional space including a first virtual object which is a real object and a second virtual object which is not a real object, comprising:

a first shadow region calculation unit configured to calculate a region in the third-dimensional space in which a shadow of the first virtual object is cast; and

a pixel value calculation unit configured to add a value of an influence of a shadow of the second virtual object to values of pixels in a region except for the shadow region calculated using the first shadow region calculation unit.