IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD AND STORAGE MEDIUM

• US 20150228093A1
• Filed: 02/04/2015
• Published: 08/13/2015
• Est. Priority Date: 02/07/2014
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Miyasa, Kazuhiro, Miyasato, Takuro
• Granted Patent: US 9,691,150 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/10088 Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/30068 Mammography; Breast
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

An image processing apparatus includes: an image obtaining unit configured to obtain a first image by imaging an object using a first imaging apparatus, a second image by imaging the object using a second imaging apparatus, a third image by imaging the object using an image capturing unit whose position is associated with the second imaging apparatus; and an alignment unit configured to align the object in the first image with the object in the second image so that observation information of the object in the first image matches observation information of the object in the third image.

Claims

1. An image processing apparatus comprising:

an image obtaining unit configured to obtain a first image by imaging an object using a first imaging apparatus, a second image by imaging the object using a second imaging apparatus, a third image by imaging the object using an image capturing unit whose position is associated with the second imaging apparatus; and

an alignment unit configured to align the object in the first image with the object in the second image so that observation information of the object in the first image matches observation information of the object in the third image.

2. The apparatus according to claim 1, wherein after performing alignment so that the observation information of the object in the first image matches the observation information of the object in the third image, said alignment unit uses an alignment result to align the object in the first image with the object in the second image by comparing the observation information of the object in the first image with the observation information of the object in the second image.

3. The apparatus according to claim 2,

wherein said image obtaining unit includes

a first image obtaining unit configured to obtain the first image by imaging the object in a first state using the first imaging apparatus,

a second image obtaining unit configured to obtain the second image by imaging the object in a second state different from the first state using the second imaging apparatus, and

a third image obtaining unit configured to obtain the third image by imaging the object in an early stage in the second state using the image capturing unit, and

wherein after aligning the first image with the third image by comparing the first image with the third image, said alignment unit uses an alignment result to align the first image with the second image by comparing the first image with the second image.

4. The apparatus according to claim 2, wherein

the first imaging apparatus and the second imaging apparatus are imaging apparatuses configured to capture three-dimensional images of the object as the first image and the second image, respectively,

the image capturing unit is a camera, and

said alignment unit aligns the first image with the second image by comparison of surface shape information of the object between the first image and the third image and comparison of internal information of the object between the first image and the third image.

5. The apparatus according to claim 1, wherein

the image capturing unit is an infrared camera configured to capture an infrared camera image visualizing a blood vessel area near a surface of the object, and

said alignment unit performs the alignment by performing comparison of the blood vessel area near the surface of the object between the first image and the infrared camera image.

6. The apparatus according to claim 1, wherein

the image capturing unit is a distance image camera configured to capture a three-dimensional shape of a surface of the object as a distance image, and

said alignment unit performs the alignment by performing comparison of a three-dimensional surface shape of the object between the first image and the distance image.

7. The apparatus according to claim 1, wherein the first imaging apparatus is an imaging apparatus configured to capture a three-dimensional image of the object as the first image,

said image processing apparatus further comprising:

an information obtaining unit configured to obtain information indicating a surface position of the object from the three-dimensional image; and

a projected image generation unit configured to generate, based on the information indicating the surface position, a projected image of the three-dimensional image when seen from a viewpoint of the image capturing unit,

wherein said alignment unit aligns the object in the first image with the object in the second image so that the projected image matches the observation information of the object in the third image.

8. The apparatus according to claim 7, further comprising a deformed image generation unit configured to deform the first image captured in the first state into the first image in the second state by using a result of aligning the first image with the third image,

wherein said deformed image generation unit generates the first image deformed from the first state to the second state by using an alignment parameter for aligning the first image with the second image, and

said projected image generation unit generates a projected image seen from the viewpoint of the image capturing unit by using the deformed first image.

9. The apparatus according to claim 8, wherein the alignment parameter includes a parameter representing a position/orientation of the object and deformation of the object.

10. The apparatus according to claim 8, wherein said deformed image generation unit generates, based on a plurality of candidate values of the alignment parameter, a plurality of first images deformed in the second state.

11. The apparatus according to claim 10, further comprising an evaluation unit configured to obtain similarities between the second image and the plurality of first images deformed in the second state,

wherein said alignment unit performs the alignment by comparing the first images with the second image using the similarities.

12. The apparatus according to claim 11, wherein said evaluation unit selects one alignment parameter from the plurality of candidate values according to evaluation of the similarities.

13. The apparatus according to claim 1, further comprising an image display unit configured to display the first image and the second image on a display unit side by side.

15. A computer-readable storage medium storing a program for causing a computer to function as each unit of an image processing apparatus defined in claim 1.

14. An image processing method for an image processing apparatus, comprising:

an image obtaining step of obtaining a first image by imaging an object using a first imaging apparatus, a second image by imaging the object using a second imaging apparatus, a third image by imaging the object using an image capturing unit whose position is associated with the second imaging apparatus; and

an alignment step of aligning the object in the first image with the object in the second image so that observation information of the object in the first image matches observation information of the object in the third image.