IMAGE PROCESSING METHOD AND DEVICE, AND PROGRAM

• US 20130223759A1
• Filed: 01/30/2013
• Published: 08/29/2013
• Est. Priority Date: 02/28/2012
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Nishiyama, Tomohiro
• Granted Patent: US 9,208,396 B2
• Time in Patent Office: Days
• US Class Current: 382/284
• CPC Class Codes: G06T 2207/10052 Images from lightfield camera
• CPC Class Codes: G06T 2207/10148 Varying focus
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

In image processing of multi-viewpoint image data including image data captured with different focal lengths, an image of high quality, distance information with high precision, etc., are obtained by utilizing image data with different angles of view (focal lengths). An image processing device for generating combined image data using multi-viewpoint image data including image data acquired with different focal lengths, includes a resolution converting unit configured to perform resolution conversion for at least part of image data in multi-viewpoint image data in accordance with a focal length to be output and an image combining unit configured to generate combined image data with the focal length to be output using the resolution-converted image data.

Claims

1. An image processing device for generating combined image data using multi-viewpoint image data including image data acquired with different focal lengths, the image processing device comprising:

a resolution converting unit configured to perform resolution conversion for at least part of image data in multi-viewpoint image data in accordance with a focal length to be output; and

an image combining unit configured to generate combined image data with the focal length to be output using the resolution-converted image data.

2. The image processing device according to claim 1, wherein

the at least part of image data is image data acquired with a focal length shorter than the focal length to be output, and

the resolution converting unit performs resolution conversion accompanied by enlargement for the image data acquired with the shorter focal length.

3. The image processing device according to claim 1, wherein

the at least part of image data is image data acquired with a focal length greater than the focal length to be output, and

the resolution converting unit performs resolution conversion accompanied by reduction for the image data acquired with the greater focal length.

4. The image processing device according to claim 1, wherein

the image combining unit generates the combined image data by applying a filter to the multi-viewpoint image data to weight and add the multi-viewpoint image data based on information of a focus position to be output.

5. The image processing device according to claim 1, wherein

the multi-viewpoint image data is image data acquired by an image capturing device having two or more kinds of focal lengths and capable of simultaneously capturing a plurality of images from different viewpoints.

6. The image processing device according to claim 5, wherein

the image capturing device capable of simultaneously capturing a plurality of images from different viewpoints is a camera array with different focal points or a Plenoptic camera with different focal points.

7. An image processing device for deriving a distance to a subject using multi-viewpoint image data including image data acquired with different focal lengths, the image processing device comprising:

a parallax deriving unit configured to derive a parallax between images acquired with the different focal lengths; and

a distance estimating unit configured to estimate a distance to a subject based on the derived parallax.

8. The image processing device according to claim 7, further comprising a resolution converting unit configured to perform resolution conversion for at least part of image data in the multi-viewpoint image data in accordance with a focal length to be output, wherein

the parallax deriving unit derives a parallax using the resolution-converted image data.

9. An image processing device for generating combined image data using multi-viewpoint image data including image data acquired with different focal lengths, the image processing device comprising:

a distance estimating unit configured to estimate a distance to a subject using image data acquired with a first focal length; and

an image combining unit configured to generate combined image data using the estimated distance and image data acquired with a second focal length different from the first focal length.

10. An image processing method for generating combined image data using multi-viewpoint image data including image data captured with different focal lengths, the image processing method comprising the steps of:

performing resolution conversion for at least part of image data in the multi-viewpoint image data in accordance with a focal length to be output; and

generating combined image data with the focal length to be output using the resolution-converted image data.

13. A program stored in a non-transitory computer readable storage medium for causing a computer to perform the image processing method according to claim 10.

11. An image processing method for estimating a distance to a subject using multi-viewpoint image data including image data acquired with different focal lengths, the image processing method comprising the steps of:

deriving a parallax between images acquired with the different focal lengths; and

estimating a distance to a subject based on the derived parallax.

14. A program stored in a non-transitory computer readable storage medium for causing a computer to perform the image processing method according to claim 11.

12. An image processing method for generating combined image data using multi-viewpoint image data including image data captured with different focal lengths, the image processing method comprising the steps of:

estimating a distance to a subject using image data acquired with a first focal length; and

generating combined image data using the estimated distance and image data acquired with a second focal length different from the first focal length.

15. A program stored in a non-transitory computer readable storage medium for causing a computer to perform the image processing method according to claim 12.