Image processing apparatus, image processing method, image processing program, and image pickup apparatus acquiring a focusing distance from a plurality of images

• US 9,621,786 B2
• Filed: 02/12/2015
• Issued: 04/11/2017
• Est. Priority Date: 05/17/2012
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Takahashi, Takahiro
• Primary Examiner(s): Villecco, John
• Application Number:
  US14/620,408
• Publication Number:
  US 20150156404A1
• Time in Patent Office: 789 Days
• Field of Search: 382106, 382107, 382255
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: H04N 23/676 Bracketing for image capture at varying focusing conditions
• CPC Class Codes: H04N 23/959 by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics

Abstract

An image processing apparatus that acquires a focusing distance from a plurality of images each having different degrees of blur, includes: a misalignment detection unit that detects misalignment among the plurality of images; a distance information acquisition unit that acquires, based on the plurality of images, distance information, which is information to indicate a focusing distance in an area of the images; and a reliability information acquisition unit that creates, based on the detected misalignment, reliability information, which is information to indicate reliability of the focusing distance acquired for each area, wherein the distance information acquisition unit changes the acquired distance information based on the reliability information.

Claims

1. A method for generating a depth map comprising the steps of:

generating, in a first generating step, a depth map from an image signal acquired by a passive method; and

generating, in a second generating step, from the image signal, an integrated reliability map representing the reliability of distance information of the depth map,

wherein the second generating step comprises the steps of;

generating a first reliability map by performing a first process on the image signal,

generating a second reliability map by performing a second process, which is different from the first process, on the image signal, and

integrating the first reliability map and the second reliability map to generate the integrated reliability map, and

wherein the first reliability map, the second reliability map, and the integrated reliability map have the same resolution.

2. The method according to claim 1, wherein the first reliability map and the second reliability map are generated by using variable weights.

3. The method according to claim 2, wherein the weights are changed based on a photographing mode used when the image signal is acquired.

4. The method according to claim 1, further comprising acquiring the distance information based on a correlation between a first image signal and a second image signal, wherein the image signal includes the first image signal and the second image signal.

5. The method according to claim 1, wherein the distance information is information related to a distance to a subject.

6. The method according to claim 1, wherein the image signal includes a first image signal and a second image signal, and the integrated reliability map is generated by analyzing the first image signal or the second image signal, and the second reliability map is generated based on a misalignment of the first image signal and the second image signal.

7. The method according to claim 1, wherein the first reliability map is generated using first evaluation criteria, and the second reliability map is generated using second evaluation criteria, which is different from the first evaluation criteria.

8. An apparatus that generates a depth map comprising:

circuitry, the circuitry configured to;

generate a depth map from an image signal acquired by a passive method; and

generate, from the image signal, an integrated reliability map representing the reliability of distance information of the depth map,

wherein, when generating the integrated reliability map, the circuitry is further configured to;

generate a first reliability map by performing a first process on the image signal,

generate a second reliability map by performing a second process, which is different from the first process, on the image signal, and

integrate the first reliability map and the second reliability map to generate the integrated reliability map, and

wherein the first reliability map, the second reliability map, and the integrated reliability map have the same resolution.

9. The apparatus according to claim 8, wherein the circuitry integrates the first reliability map and the second reliability map by using variable weights.

10. The apparatus according to claim 9, wherein the circuitry changes the weights based on a photographing mode used when the image signal is acquired.

11. The apparatus according to claim 8, wherein the image signal includes a first image signal and a second image signal, and the circuitry acquires the distance information based on a correlation between the first image signal and the second image signal.

12. The apparatus according to claim 8, wherein the distance information is information related to a distance to a subject.

13. The apparatus according to claim 8, wherein the image signal includes a first image signal and a second image signal, and the circuitry generates the integrated reliability map by analyzing the first image signal or the second image signal, and generates the second reliability map based on a misalignment of the first image signal and the second image signal.

14. The apparatus according to claim 8, wherein the circuitry generates the first reliability map using first evaluation criteria, and generates the second reliability map using second evaluation criteria, which is different from the first evaluation criteria.

15. A non-transitory computer-readable storage medium recording a computer program for causing a computer to perform a method for generating a depth map, the method comprising the steps of:

generating, in a first generating step, a depth map from an image signal acquired by a passive method; and

generating, in a second generating step, from the image signal, an integrated reliability map representing the reliability of distance information of the depth map, wherein the second generating step comprises the steps of;

generating a first reliability map by performing a first process on the image signal,

generating a second reliability map by performing a second process, which is different from the first process, on the image signal, and

integrating the first reliability map and the second reliability map to generate the integrated reliability map, and

wherein the first reliability map, the second reliability map, and the integrated reliability map have the same resolution.

16. An apparatus that generates a depth map comprising:

depth map generating means for generating a depth map from an image signal acquired by a passive method; and

reliability map generating means for generating, from the image signal, an integrated reliability map representing the reliability of distance information of the depth map,

wherein the reliability map generating means;

generates a first reliability map by performing a first process on the image signal,

generates a second reliability map by performing a second process, which is different from the first process, on the image signal, and

integrates the first reliability map and the second reliability map to generate the integrated reliability map.