IMAGE PROCESSING APPARATUS, IMAGE PICKUP APPARATUS, IMAGE PROCESSING METHOD, AND NON-TRANSITORY COMPUTER-READABLE STORAGE MEDIUM FOR SYNTHESIZING IMAGES

• US 20160205380A1
• Filed: 01/04/2016
• Published: 07/14/2016
• Est. Priority Date: 01/09/2015
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Nishiyama, Tomohiro, Inoue, Chiaki
• Granted Patent: US 9,992,478 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

An image processing apparatus includes an image capturing condition acquirer (260) configured to acquire image capturing condition information of a plurality of images having angles of view different from each other, a distance information acquirer (230) configured to acquire distance information of an object included in the plurality of images, an angle of view information acquirer (240) configured to acquire angle of view information, and an image generator (224) configured to generate an image corresponding to the angle of view information from the plurality of images based on the image capturing condition information, the distance information, and the angle of view information, and the distance information is object distance information at each position in an image.

Claims

1. An image processing apparatus comprising:

an image capturing condition acquirer configured to acquire image capturing condition information of a plurality of images having angles of view different from each other;

a distance information acquirer configured to acquire distance information of an object included in the plurality of images;

an angle of view information acquirer configured to acquire angle of view information; and

an image generator configured to generate an image corresponding to the angle of view information from the plurality of images based on the image capturing condition information, the distance information, and the angle of view information,

wherein the distance information is object distance information at each position in an image.

2. The image processing apparatus according to claim 1, wherein the image generator is configured to generate the image corresponding to the angle of view information from the plurality of images based on a deterioration amount of resolution calculated by using the image capturing condition information, the distance information, and the angle of view information.

3. The image processing apparatus according to claim 1, further comprising a position controller configured to control positions of viewpoints for the plurality of images to coincide with each other.

4. The image processing apparatus according to claim 1, further comprising a weight determiner configured to determine a weight based on the image capturing condition information, the distance information, and the angle of view information,

wherein the image generator is configured to generate the image corresponding to the angle of view information based on the weight.

5. The image processing apparatus according to claim 4, wherein the weight determiner is configured to:

calculate a deterioration amount of a resolution of each of the plurality of images based on the image capturing condition information, the distance information, and the angle of view information, and

determine the weight based on the deterioration amount of the resolution.

6. The image processing apparatus according to claim 4, further comprising a mode selector configured to select an image capturing mode,

wherein the image capturing mode includes a resolution priority mode and an S/N priority mode, and

wherein the weight determiner is configured to determine the weight depending on the image capturing mode selected by the mode selector.

7. The image processing apparatus according to claim 4, wherein the weight determiner is configured to determine the weight based on a feature amount of each of the plurality of images.

8. The image processing apparatus according to claim 7, wherein the feature amount of each of the plurality of images is an edge intensity of each of the images.

9. The image processing apparatus according to claim 1, further comprising a blur adder configured to add blur to each of the plurality of images based on the image capturing condition information, the distance information, and the angle of view information,

wherein the image generator is configured to generate the image corresponding to the angle of view information from the plurality of images including the blur added by the blur adder.

10. The image processing apparatus according to claim 9, wherein the blur adder is configured to:

calculate a deterioration amount of a resolution of each of the plurality of images based on the image capturing condition information, the distance information, and the angle of view information, and

add the blur based on the deterioration amount of the resolution.

11. The image processing apparatus according to claim 10, wherein the blur adder is configured to increase the blur with decreasing the deterioration amount of the resolution.

12. The image processing apparatus according to claim 9, wherein the blur adder is configured to add the blur so as to reduce a difference of depths of field in the plurality of images.

13. The image processing apparatus according to claim 12, wherein the blur adder is configured to add the blur such that the depths of field in the plurality of images coincide with each other.

14. The image processing apparatus according to claim 1, wherein the plurality of images include a plurality of images acquired at an identical angle of view.

15. The image processing apparatus according to claim 1, wherein the distance information acquirer is configured to acquire the distance information based on the plurality of images having viewpoints different from each other.

16. An image pickup apparatus comprising:

an image pickup device configured to acquire a plurality of images having angles of view different from each other;

an image capturing condition acquirer configured to acquire image capturing condition information of the plurality of images;

a distance information acquirer configured to acquire distance information of an object included in the plurality of images;

an angle of view information acquirer configured to acquire angle of view information; and

an image generator configured to generate an image corresponding to the angle of view information from the plurality of images based on the image capturing condition information, the distance information, and the angle of view information,

wherein the distance information is object distance information at each position in an image.

17. An image processing method comprising the steps of:

acquiring image capturing condition information of a plurality of images having angles of view different from each other;

acquiring distance information of an object included in the plurality of images;

acquiring angle of view information; and

generating an image corresponding to the angle of view information from the plurality of images based on the image capturing condition information, the distance information, and the angle of view information,

wherein the distance information is object distance information at each position in an image.

18. A non-transitory computer-readable storage medium storing a program which causes a computer to execute a process comprising the steps of:

acquiring image capturing condition information of a plurality of images having angles of view different from each other;

acquiring distance information of an object included in the plurality of images;

acquiring angle of view information; and

generating an image corresponding to the angle of view information from the plurality of images based on the image capturing condition information, the distance information, and the angle of view information,

wherein the distance information is object distance information at each position in an image.