IMAGE PROCESSING APPARATUS AND METHOD

• US 20080284864A1
• Filed: 08/08/2007
• Published: 11/20/2008
• Est. Priority Date: 08/11/2006
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Iwase, Yoshihiko, Suzuki, Masahiro, Satoh, Kiyohide, Kotake, Daisuke, Takemoto, Rika, Nakazawa, Toshihiro
• Granted Patent: US 8,144,238 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning

Abstract

In position and orientation measurement based on natural features, erroneous detection of the natural features is prevented when an observation target object is occluded by another object, and registration stability is improved. To this end, an occluding object that can occlude the observation target object is defined, and an occluding region where the occluding object occludes the observation target object is detected in an input captured image. Image features of the observation target object are detected in a region of the captured image other than the detected occluding region. Therefore, the position or orientation of an image pickup apparatus that captured the captured image, or the position or orientation of the observation target object in the captured image are calculated.

Claims

1. An image processing apparatus comprising:

an object defining unit configured to define a specific object that can occlude a target object to be observed;

an image obtaining unit configured to obtain a captured image captured by an image pickup apparatus;

a region detector configured to detect a region including the specific object from the obtained captured image;

an image feature detector configured to detect image features of the target object from the captured image; and

a calculating unit configured to calculate a position or orientation of the image pickup apparatus that captured the captured image or a position or orientation of the target object in the captured image on the basis of the image features detected by the image feature detector,

wherein the image processing apparatus is configured not to take into account features from the detected region in determining the position and/or orientation of the image-pickup apparatus that captured the image or a position and/or orientation of the target object in the captured image.

2. The image processing apparatus according to claim 1, wherein the region detector is configured to detect a region comprising a region where the specific object occludes the target object, or a region where the specific object occludes the target object and a region adjacent thereto.

3. The image processing apparatus according to claim 1, wherein the image feature detector is configured to detect the image features on the basis of the detected region by detecting in a region of the captured image other than the detected region.

4. The image processing apparatus according to claim 1, wherein the calculating unit is configured to calculate, on the basis of the detected region, the position and/or orientation of the image pickup apparatus that captured the captured image or the position and/or orientation of the target object in the captured image on the basis of the image features detected in a region of the captured image other than the detected region.

5. The image processing apparatus according to claim 1, wherein the object defining unit is configured to define the specific object in terms of image feature information, and

the region detector is configured to detect the region from the captured image on the basis of the defined image feature information.

6. The image processing apparatus according to claim 5, wherein the image feature information defining the specific object comprises a color or a pattern.

7. The image processing apparatus according to claim 6, wherein the specific object comprises human skin, and the color comprises the color of the skin.

8. The image processing apparatus according to claim 5, wherein the image feature information defining the specific object comprises features representing a human face region on an image.

9. The image processing apparatus according to claim 1, wherein the object defining unit is configured to define a three-dimensional shape of the specific object,

the image processing apparatus further comprising:

a first rough position and orientation obtaining unit configured to obtain a rough position and rough orientation of the image pickup apparatus; and

a second rough position and orientation obtaining unit configured to obtain a rough position of the specific object or a rough position and rough orientation of the specific object, and

the region detector detects the region including the specific object from the captured image on the basis of the rough position and rough orientation of the image pickup apparatus obtained by the first rough position and orientation obtaining unit, the rough position of the specific object or the rough position and rough orientation of the specific object obtained by the second rough position and orientation obtaining unit, and the three-dimensional shape of the specific object.

10. The image processing apparatus according to claim 1, further comprising:

a virtual-space image generator configured to generate a virtual-space image on the basis of a calculation result of the calculating unit; and

an image synthesizing unit configured to synthesize the captured image and the virtual-space image,

wherein an image in which the captured image and the virtual-space image are synthesized is presented to a user.

11. The image processing apparatus according to claim 10, wherein the virtual-space image is not synthesized with the region detected by the region detector.

12. The image processing apparatus according to claim 1, wherein the image features comprise line features or point features.

13. An image processing method comprising:

a holding step of holding definition information defining a specific object that can occlude a target object to be observed;

an image obtaining step of obtaining a captured image captured by an image pickup apparatus;

a region detecting step of detecting a region including the specific object from the obtained captured image;

an image feature detecting step of detecting image features of the target object from the captured image; and

a calculating step of calculating a position and/or orientation of the image pickup apparatus that captured the captured image or a position and/or orientation of the target object in the captured image on the basis of the image features detected in the image feature detecting step,

wherein the method does not take into account features from the detected region in determining the position and/or orientation of the image-pickup apparatus that captured the image or a position and/or orientation of the target object in the captured image.

14. A computer program for implementing the image processing method according to claim 13 by using a computer.