Image capturing apparatus, image capturing method, and storage medium using compressive sensing

• US 10,154,216 B2
• Filed: 05/15/2017
• Issued: 12/11/2018
• Est. Priority Date: 05/17/2016
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Hiasa, Norihito
• Primary Examiner(s): Fosselman, Joel W
• Application Number:
  US15/594,943
• Publication Number:
  US 20170339363A1
• Time in Patent Office: 575 Days
• Field of Search: 3482211
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/743 Bracketing, i.e. taking a series of images with varying exposure conditions
• CPC Class Codes: H04N 23/75 by influencing optical camera components
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/75 Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 25/771 comprising storage means other than floating diffusion
• CPC Class Codes: H04N 25/78 Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters

Abstract

An image capturing apparatus acquires a light intensity distribution in an object space through image capturing using a plurality of pixels, sets a coefficient distribution that is a distribution of a coefficient corresponding to each of the plurality of pixels and is to be applied to the light intensity distribution, acquires object space information that is information on the object space and different from the light intensity distribution, sets first pixel group and second pixel groups that are different from each other in the plurality of pixels based on the object space information, and generates a combined image by combining a plurality of light intensity distributions obtained by applying the coefficient distributions to each of the plurality of pixels obtained with a plurality of image capturing conditions.

Claims

1. An image capturing apparatus comprising:

a light intensity distribution acquirer configured to acquire a light intensity distribution in an object space through image capturing using a plurality of pixels;

a coefficient distribution setter configured to set a coefficient distribution that is a distribution of a coefficient corresponding to each of the plurality of pixels and is to be applied to the light intensity distribution, such that the coefficient distribution is different for each image capturing condition;

an information acquirer configured to acquire object space information that is information on the object space and different from the light intensity distribution;

a pixel group setter configured to set a first pixel group and a second pixel group that are different from each other in the plurality of pixels, based on the object space information; and

an image generator configured to generate a combined image by combining a plurality of light intensity distributions obtained by applying the coefficient distribution to each of a plurality of light intensity distributions obtained on a plurality of image capturing conditions,

wherein the coefficient distribution setter sets a first coefficient distribution as at least one of the plurality of coefficient distributions applied to each of the plurality of light intensity distributions, such that fundamental statistics of the coefficient distribution for the first pixel group is different from that for the second pixel group in the first coefficient distribution.

2. The image capturing apparatus according to claim 1, wherein the object space information includes information on a distribution of at least one of a light amount of the object space, an edge of the object space, a polarization of the object space, a distance of the object space, and a motion of the object space relative to the image capturing apparatus.

3. The image capturing apparatus according to claim 1, wherein the fundamental statistics includes at least one of a representative value and a dissemination of the coefficient.

4. The image capturing apparatus according to claim 1, wherein the fundamental statistics includes at least one of an average value, a median, a mode, a contrast ratio, and a dispersion of the coefficient.

5. The image capturing apparatus according to claim 1, wherein an average value or dispersion of the coefficient corresponding to the first pixel group is different from that corresponding to the second pixel group in the first coefficient distribution.

6. The image capturing apparatus according to claim 1, wherein the fundamental statistics of a coefficient distribution for the first pixel group and that for the second pixel group in the first coefficient distribution are different from each other by an amount at least one-tenth as large as a smaller one of the fundamental statistics.

7. The image capturing apparatus according to claim 1, wherein the first pixel group has a larger average value of the coefficient as the fundamental statistics in the first coefficient distribution among the first and second pixel groups, and

wherein the plurality of coefficient distributions include a second coefficient distribution different from the first coefficient distribution, and an average value of the coefficient for the second pixel group is larger than that for the first pixel group in the second coefficient distribution.

8. The image capturing apparatus according to claim 1, wherein the following conditions are satisfied,

9. The image capturing apparatus according to claim 1, wherein an average value ratio and a dispersion ratio are defined as ratios made by dividing an average value and a dispersion of the coefficient corresponding to each pixel groups in the first coefficient distribution by a corresponding one of the average value and the dispersion of the coefficient corresponding to each pixel groups in a second coefficient distribution different from the first coefficient distribution, and

wherein the first pixel group and second pixel group are different from each other in average value ratio or dispersion ratio.

10. The image capturing apparatus according to claim 1, wherein in the first coefficient distribution, both an average value and a dispersion of the coefficient corresponding to the first pixel group are larger or smaller than those corresponding to the second pixel group.

11. The image capturing apparatus according to claim 1, wherein a same intensity pixel group is defined as a pixel group in the plurality of pixels, which has a difference among the plurality of light intensity distribution obtained on the plurality of image capturing conditions in the plurality of pixels, smaller than a threshold, and

wherein the following conditions are satisfied,

12. The image capturing apparatus according to claim 1, wherein the pixel group setter sets the first pixel group or second pixel group on a specific image capturing condition in the plurality of image capturing conditions to a pixel group used to obtain the light intensity distribution with a higher resolution than that on another image capturing condition or with another pixel group, and

wherein the coefficient setter sets the first coefficient distribution to the light intensity distribution obtained on the specific image capturing condition.

13. An image processing apparatus installed in or separate from an image capturing apparatus,

wherein the image capturing comprising:

a light intensity distribution acquirer configured to acquire a light intensity distribution in an object space through image capturing using a plurality of pixels;

a coefficient distribution setter configured to set a coefficient distribution that is a distribution of a coefficient corresponding to each of the plurality of pixels and is to be applied to the light intensity distribution, such that the coefficient distribution is different for each image capturing condition;

an information acquirer configured to acquire object space information that is information on the object space and different from the light intensity distribution;

a pixel group setter configured to set a first pixel group and a second pixel group that are different from each other in the plurality of pixels, based on the object space information; and

an image generator configured to generate a combined image by combining a plurality of light intensity distributions obtained by applying the coefficient distribution to each of a plurality of light intensity distributions obtained on a plurality of image capturing conditions,

wherein the coefficient distribution setter sets a first coefficient distribution as at least one of the plurality of coefficient distributions applied to each of the plurality of light intensity distributions, such that fundamental statistics of the coefficient distribution for the first pixel group is different from that for the second pixel group in the first coefficient distribution, and

wherein the image processing apparatus comprises an image estimator configured to generate image data through an estimation process, which corresponds to at least one of the plurality of image capturing conditions based on the combined image.

14. An image capturing method comprising:

a step of acquiring a light intensity distribution in an object space through image capturing using a plurality of pixels;

a coefficient distribution setting step configured to set a coefficient distribution that is a distribution of a coefficient corresponding to each of the plurality of pixels and is to be applied to the light intensity distribution, such that the coefficient distribution is different for each image capturing condition;

a step of acquiring object space information that is information on the object space and different from the light intensity distribution;

a step of setting a first pixel group and a second pixel group that are different from each other in the plurality of pixels, based on the object space information; and

a step of generating a combined image by combining a plurality of light intensity distributions obtained by applying the coefficient distribution to each of a plurality of light intensity distributions obtained on a plurality of image capturing conditions,

wherein the coefficient distribution setting step sets a first coefficient distribution as at least one of the plurality of coefficient distributions applied to each of the plurality of light intensity distributions, such that fundamental statistics of the coefficient distribution for the first pixel group is different from that for the second pixel group in the first coefficient distribution.

15. A non-transitory computer readable storage medium storing an image capturing program that enables a computer to execute an image capturing method, the image capturing method comprising:

a step of acquiring a light intensity distribution in an object space through image capturing using a plurality of pixels;

a coefficient distribution setting step configured to set a coefficient distribution that is a distribution of a coefficient corresponding to each of the plurality of pixels and is to be applied to the light intensity distribution, such that the coefficient distribution is different for each image capturing condition;

a step of acquiring object space information that is information on the object space and different from the light intensity distribution;

a step of setting a first pixel group and a second pixel group that are different from each other in the plurality of pixels, based on the object space information; and

a step of generating a combined image by combining a plurality of light intensity distributions obtained by applying the coefficient distribution to each of a plurality of light intensity distributions obtained on a plurality of image capturing conditions,

wherein the coefficient distribution setting step sets a first coefficient distribution as at least one of the plurality of coefficient distributions applied to each of the plurality of light intensity distributions, such that fundamental statistics of the coefficient distribution for the first pixel group is different from that for the second pixel group in the first coefficient distribution.