IMAGE CAPTURING APPARATUS, IMAGE PROCESSING APPARATUS, AND METHOD THEREOF

• US 20120307093A1
• Filed: 05/14/2012
• Published: 12/06/2012
• Est. Priority Date: 05/31/2011
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Miyoshi, Ai
• Granted Patent: US 8,970,714 B2
• Time in Patent Office: Days
• US Class Current: 348/218.1
• CPC Class Codes: G06T 2200/21 involving computational photography
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/959 by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

An image processing apparatus and method relate to an imaging process which is applied to image data captured by an image capturing apparatus such as a multi-eye camera, multi-view camera, and the like. The image processing apparatus and method generates synthetic image data which focused on a curved focus surface by compositing multi-view image data captured from multi-viewpoints.

Claims

1. An image processing apparatus comprising an image generation unit configured to composite multi-view image data obtained by capturing from a plurality of viewpoints, thereby generating synthetic image data having focus on a curved focus surface,

wherein the image generation unit is implemented using a processor.

2. The apparatus according to claim 1, wherein the image generation unit generates synthetic image data having focus on the curved focus surface in each pixel of an image represented by the synthetic image data.

3. The apparatus according to claim 1, further comprising:

an acquisition unit configured to acquire focus coordinates representing a plurality of positions to be in focus in an image represented by the synthetic image data; and

a surface generation unit configured to generate the curved focus surface from the acquired focus coordinates.

4. The apparatus according to claim 3, wherein the surface generation unit generates the curved focus surface from a distance to an object and the acquired focus coordinates.

5. The apparatus according to claim 4, wherein the surface generation unit sets a curve having an order corresponding to the number of acquired focus coordinates and passing through the focus coordinates, and expands the curve to obtain the focus surface.

6. The apparatus according to claim 5, wherein the surface generation unit limits the order not to exceed a predetermined number.

7. The apparatus according to claim 4, further comprising an input unit configured to input a depth of field of the multi-view image data,

wherein the surface generation unit sets a curve having an order corresponding to the number of groups obtained by grouping the focus coordinates based on the depth of field and passing through representative focus coordinates of the groups, and expands the curve to obtain the focus surface.

8. The apparatus according to claim 4, further comprising:

an input unit configured to input the multi-view image data and information representing a capturing position of the multi-view image data; and

an estimation unit configured to estimate the distance to the object from the multi-view image data and the information representing the capturing position.

9. The apparatus according to claim 8, wherein the image generation unit comprises:

a setting unit configured to set a weighting coefficient of each of the multi-view image data;

a calculation unit configured to calculate a shift amount of an image represented by each of the multi-view image data based on the information representing the capturing position and information representing the focus surface; and

a composition unit configured to generate the synthetic image data by shifting each of the multi-view image data by the shift amount, multiplying each of the shifted multi-view image data by the weighting coefficient, and compositing the multi-view image data each multiplied by the weighting coefficient.

10. An image capturing apparatus comprising:

a plurality of capturing units which capture an object from a plurality of viewpoints; and

an image processing apparatus described in claim 1.

11. An image processing method comprising:

using a processor to perform the step of compositing multi-view image data obtained by capturing from a plurality of viewpoints so as to generate synthetic image data having focus on a curved focus surface.

12. A non-transitory computer readable medium comprising a computer-executable program for causing a computer to perform an image processing method, the method comprising the step of compositing multi-view image data obtained by capturing from a plurality of viewpoints so as to generate synthetic image data having focus on a curved focus surface.