Image processing apparatus, image processing apparatus control method, image pickup apparatus, and image pickup apparatus control method

• US 10,432,852 B2
• Filed: 08/07/2015
• Issued: 10/01/2019
• Est. Priority Date: 08/20/2014
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Nakayama, Fumitaka
• Primary Examiner(s): Haskins, Twyler L
• Assistant Examiner(s): Tejano, Dwight Alex C
• Application Number:
  US14/821,091
• Publication Number:
  US 20160057341A1
• Time in Patent Office: 1,516 Days
• CPC Class Codes: G06T 2207/10148 Varying focus
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals
• CPC Class Codes: H04N 23/672 based on the phase difference signals
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/95 Computational photography systems, e.g. light-field imaging systems
• CPC Class Codes: H04N 25/704 Pixels specially adapted for focusing, e.g. phase difference pixel sets
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

An image processing apparatus including an acquiring unit configured to acquire an image signal of a plurality of images obtained by picking up an object image that is formed by a photographing optical system; a determining unit configured to determine a defocus amount of the object image by using the image signal of the plurality of images; and a conversion unit configured to perform, on the defocus amount, gradation conversion that has at least one conversion characteristic out of a plurality of different conversion characteristics, and to output information that is based on the defocus amount converted by the gradation conversion.

Claims

1. An image processing apparatus, comprising:

one or more processors; and

a memory storing instructions which, when the instructions are executed by the one or more processors, cause the image processing apparatus to function as;

an acquiring unit configured to acquire a plurality of image signals picked up by an image pickup unit;

a determining unit configured to determine distance information of the object image by using the plurality of image signals; and

a conversion unit configured to perform, on the distance information, gradation conversion that has at least one conversion characteristic out of a plurality of different conversion characteristics, and to output information that is based on the distance information converted by the gradation conversion,

wherein the conversion unit selects the one conversion characteristic out of the plurality of different conversion characteristics based on an application to which the information is output, and

wherein the conversion unit determines a gradation resolution of the output of the conversion unit based on the conversion characteristic.

2. The image processing apparatus according to claim 1, wherein

the plurality of different conversion characteristics comprise conversion characteristics with which gradation conversion varies depending on an object distance.

3. The image processing apparatus according to claim 1, wherein

the plurality of different conversion characteristics comprise conversion characteristics with which gradation conversion varies depending on an F-value of a lens of the photographing optica system.

4. The image processing apparatus according to claim 1, wherein

the plurality of different conversion characteristics comprise conversion characteristics with which gradation conversion varies depending on a base length in the image pickup of the object image.

5. The image processing apparatus according to claim 1, wherein

the plurality of different conversion characteristics comprise conversion characteristics with which gradation conversion varies depending on ISO sensitivity in the image pickup of the object image.

6. The image processing apparatus according to claim 1, wherein

the conversion unit selects the one conversion characteristic out of the plurality of different conversion characteristics based on a photographing condition.

7. The image processing apparatus according to claim 1,

wherein the conversion unit comprises a generating unit configured to generate information of an object distance from the distance information, and

wherein the conversion unit converts the object distance based on the one conversion characteristic out of the plurality of different conversion characteristics.

8. The image processing apparatusaccording to claim 7, wherein

the conversion unit converts the distance information based on each of at least two conversion characteristics out of the plurality of different conversion characteristics, and outputs a plurality of results obtained by the conversion of the distance information.

9. The image processing apparatus according to claim 7, further comprising:

an image dividing unit configured to acquire a plurality of divided images by dividing the plurality of image signals;

a deformation processing unit configured to deform each of the plurality of divided images based on the output of the conversion unit; and

an image synthesis unit configured to combine the plurality of divided images on which the deformation by the deformation processing unit has been performed.

10. The image processing apparatus according to claim 9,

wherein the conversion unit generates a distance map value that is obtained by converting the object distance based on the one conversion characteristic out of the plurality of different conversion characteristics, and

wherein the deformation processing unit deforms each of the plurality of divided images based on the distance map value output from the conversion unit.

11. The image processing apparatus according to claim 1, wherein

the plurality of image signals are 1) a plurality of image signals different from one another in blurring, 2) a plurality of image signals acquired by picking up object images from different regions of a pupil of a photographing optical system, or 3) a plurality of image signals acquired by picking up object images with a plurality of image pickup units.

12. The image processing apparatus according to claim 1, wherein

the distance information is based on one of 1) a value that indicates a degree of match between the plurality of images and 2) a blurring evaluation value of the plurality of images.

13. The image processing apparatus according to claim 1, wherein

the distance information is a defocus amount.

14. The image processing apparatus according to claim 1, wherein

the distance information is an object distance.

15. An image pickup apparatus, comprising:

one or more processors; and

a memory storing instructions which, when the instructions are executed by the one or more processors, cause the image processing apparatus to function as;

an image pickup unit configured to generate a plurality of image signals;

a determining unit configured to determine distance information of the object image by using the plurality of image signals of the object images that are generated by the image pickup unit;

a conversion unit configured to perform, on the distance information, gradation conversion that has at least one conversion characteristic out of a plurality of different conversion characteristics, and to output information that is based on the distance information converted by the gradation conversion;

a synthesizing unit configured to combine the plurality of image signals; and

an image processing unit configured to process the combined image signal based on the output of the conversion unit,

wherein the conversion unit selects the one conversion characteristic out of the plurality of different conversion characteristics based on a destination to which the information is output, and

wherein the image processing unit generates a processing parameter of the combined image signal based on the output of the conversion unit.

16. The image pickup apparatus according to claim 15, wherein

the image pickup unit generates a plurality of image signals of the object images on different image planes.

17. An image processing apparatus, comprising:

an acquiring unit configured to acquire a plurality of image signals picked up by an image pickup unit;

a determining unit configured to determine distance information of the object image by using the plurality of image signals; and

a conversion unit configured to perform, on the distance information, gradation conversion that has at least one conversion characteristic out of a plurality of different conversion characteristics, and to output information that is based on the distance information converted by the gradation conversion,

wherein the conversion unit selects the one conversion characteristic out of the plurality of different conversion characteristics based on a destination to which the information is output.

18. An image pickup apparatus, comprising:

an image pickup unit configured to generate a plurality of image signals;

a determining unit configured to determine distance information of the object image by using the plurality of image signals of the object images that are generated by the image pickup unit; and

a conversion unit configured to perform, on the distance information, gradation conversion that has at least one conversion characteristic out of a plurality of different conversion characteristics, and to output information that is based on the distance information converted by the gradation conversion,

wherein the conversion unit selects the one conversion characteristic out of the plurality of different conversion characteristics based on a destination to which the information is output.

19. A method for controlling an image processing apparatus, comprising:

acquiring a plurality of image signals obtained by picking up object images;

determining distance information of the object image by using the plurality of image signals; and

performing, on the distance information, gradation conversion that has at least one conversion characteristic out of a plurality of different conversion characteristics, and outputting information that is based on the distance information converted by the gradation conversion,

wherein in the gradation conversion, the one conversion characteristic out of the plurality of different conversion characteristics is selected based on an application to which the information is output, and

wherein a gradation resolution of the output of the gradation conversion is determined based on the conversion characteristic.

20. A method for controlling an image pickup apparatus,

the image pickup apparatus including an image pickup unit configured to generate a plurality of image signals,

the method comprising:

determining distance information of the object image by using a plurality of image signals of the object image that are generated by the image pickup unit;

performing, on the distance information, gradation conversion that has at least one conversion characteristic out of a plurality of different conversion characteristics, and outputting information that is based on the distance information converted by the gradation conversion;

combining, the plurality of image signals; and

processing the combined image signal based on the output of the gradation conversion,

wherein in the gradation conversion, the one conversion characteristic out of the plurality of different conversion characteristicsis selected based on a destination to which the information is output, and

wherein in processing the combined image signal, a processing parameter of the combined image signal is generated based on the output of the gradation conversion.

21. A non-transitory computer-readable storage medium having recorded thereon a program for causing a computer so that the computer executes a method for controlling an image processing apparatus, the method comprising:

acquiring a plurality of image signals obtained by picking up object images;

determining distance information of the object image by using the plurality of image signals; and

performing, on the distance information, gradation conversion that has at least one conversion characteristic out of a plurality of different conversion characteristics, and outputting information that is based on the distance information converted by the gradation conversion,

wherein in the gradation conversion, the one conversion characteristic out of the plurality of different conversion characteristics is selected based on an application to which the information is output, and

wherein a gradation resolution of the output of the gradation conversion is determined based on the conversion characteristic.

22. A non transitory computer readable storage medium having recorded thereon a program for causing a computer so that the computer executes a method for controlling an image pickup apparatus,

the image pickup apparatus including an image pickup unit configured to generate a plurality of image signals,

the method comprising:

determining distance information of the object image by using a plurality of image signals of the object image that are generated, by the image pickup unit;

performing, on the distance information, gradation conversion that has at least one conversion characteristic out of a plurality of different conversion characteristics, and outputting information that is based on the distance information converted by the gradation conversion;

combining the plurality of image signals; and

processing the combined image signal based on the output of the gradation conversion,

wherein in the gradation conversion, the one conversion characteristic out of the plurality of different conversion characteristics is selected based on a destination to which the information is output, and

wherein in processing the combined image signal, a processing parameter of the combined image signal is generated based on the output of the gradation conversion.

23. An im age pickup apparatus, comprising:

one or more processors; and

a memory storing instructions which, when the instructions are executed by the one or more processors, cause the image processing apparatus to function as;

an image pickup unit configured to generate a plurality of age signals;

a determining unit configured to determine distance information of the object image by using the plurality of image signals of the object images that are generated by the image pickup unit;

a conversion unit configured to perform, on the distance information, gradation conversion that has at least one conversion characteristic out of a plurality of different conversion characteristics, and to output information that is based on the distance information converted by the gradation conversion;

a synthesizing unit configured to combine the plurality of image signals; and

an image processing unit configured to process the combined image signal based on the output of the conversion unit,

wherein the conversion unit selects the one conversion characteristic out of the plurality of different conversion characteristics based on a destination to which the information is output,

wherein the image processing unit generates a processing parameter of the combined image signal based on the output of the conversion unit, and

wherein the image pickup unit generates at least a pair of image signals out of the object images by receiving luminous fluxes from different regions of a pupil of a photographing optical system.

24. A method for controlling an image pickup apparatus,

the image pickup apparatus including an image pickup unit configured to generate a plurality of image signals,

the method comprising:

determining distance information of the object image by using a plurality of image signals of the object image that are generated by the image pickup unit;

performing, on the distance information, gradation conversion that has at least one conversion characteristic out of plurality of different conversion characteristics, and outputting information that is based on the distance information converted by the gradation conversion;

combining the plurality of image signals; and

processing the combined image signal based on the output of the gradation conversion,

wherein in the gradation conversion, the one conversion characteristic out of the plurality of different conversion characteristics is selected based on a destination to which the information is output,

wherein in processing the combined image signal, a processing parameter of the combined image signal is generated based on the output of the gradation conversion, and

wherein the image pickup unit generates at least a pair of image signals out of the object images by receiving luminous fluxes from different regions of a pupil of a photographing optical system.

25. A non-transitory computer-readable storage medium having recorded thereon a program for causing a computer so that the computer executes a method for controlling an image pickup apparatus,

the image pickup apparatus including an image pickup unit configured to generate a plurality of image signals,

the method comprising:

determining distance information of the object image by using a plurality of image signals of the object image that are generated by the image pickup unit;

performing, on the distance information, gradation conversion that has at least one conversion characteristic out of a plurality of different conversion characteristics, and outputting information that is based on the distance information converted by the gradation conversion;

combining the plurality of image signals; and

processing the combined image signal based on the output of the gradation conversion,

wherein in the gradation conversion, the one conversion characteristic out of the plurality of different conversion characteristics is selected based on a destination to which the information is output,

wherein in processing the combined image signal, a processing parameter of the combined image signal is generated based on the output of the gradation conversion, and

wherein the image pickup unit generates at least a pair of image signals out of the object images by receiving luminous fluxes from different regions of a pupil of a photographing optical system.