IMAGE PROCESSING APPARATUS AND IMAGE PROCESSING METHOD

• US 20090102845A1
• Filed: 10/09/2008
• Published: 04/23/2009
• Est. Priority Date: 10/19/2007
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Aso, Takashi, Takemoto, Kazuki
• Granted Patent: US 9,013,483 B2
• Time in Patent Office: Days
• US Class Current: 345/426
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

An object region detection unit (130) decides the region of a physical object of interest in a physical space image. An image manipulation unit (140) performs shading processing of an inclusion region including the decided region. A rendering unit (155) arranges a virtual object in virtual space at the position and orientation of the physical object of interest and generates a virtual space image based on the position and orientation of the user'"'"'s viewpoint. A composition unit (160) generates a composite image by superimposing the virtual space image on the physical space image that has undergone the shading processing and outputs the generated composite image to an HMD (190).

Claims

1. An image processing apparatus comprising:

an unit adapted to acquire a position and orientation of a user'"'"'s viewpoint;

an unit adapted to acquire a position and orientation of a physical object of interest;

an unit adapted to acquire an image of physical space including the physical object of interest;

a decision unit adapted to decide a region of the physical object of interest in the image of the physical space;

a processing unit adapted to perform blurring processing of an inclusion region including the decided region;

an unit adapted to arrange a virtual object in virtual space at the position and orientation of the physical object of interest;

an unit adapted to generate an image of the virtual space based on the position and orientation of the viewpoint;

an unit adapted to generate a composite image by superimposing the image of the virtual space on the image of the physical space that has undergone the blurring processing; and

an unit adapted to output the composite image.

2. The apparatus according to claim 1, wherein said decision unit decides the region of the physical object of interest outside a region having a predetermined color in the image of the physical space.

3. The apparatus according to claim 1, wherein said decision unit decides a flesh color region in the image of the physical space.

4. The apparatus according to claim 1, wherein

said decision unit arranges the virtual object in the virtual space at the position and orientation of the physical object of interest and decides a region on a projection plane of the arranged virtual object, and

the inclusion region is a region obtained by expanding the decided region at a predetermined ratio.

5. The apparatus according to claim 1, further comprising an unit adapted to change a color of the region decided by said decision unit.

6. The apparatus according to claim 1, wherein the physical object of interest is a hand of the user.

7. The apparatus according to claim 1, wherein the physical object of interest is a mock-up.

8. The apparatus according to claim 1, wherein the virtual object has a shape obtained by simulating a shape of the physical object of interest.

9. An image processing method comprising:

a step of acquiring a position and orientation of a user'"'"'s viewpoint;

a step of acquiring a position and orientation of a physical object of interest;

a step of acquiring an image of physical space including the physical object of interest;

a decision step of deciding a region of the physical object of interest in the image of the physical space;

a processing step of performing blurring processing of an inclusion region including the decided region;

a step of arranging a virtual object in virtual space at the position and orientation of the physical object of interest;

a step of generating an image of the virtual space based on the position and orientation of the viewpoint;

a step of generating a composite image by superimposing the image of the virtual space on the image of the physical space that has undergone the blurring processing; and

a step of outputting the composite image.

10. A computer-readable storage medium which stores a program to cause a computer to execute an image processing method of claim 9.