Image pickup apparatus that provides a correlation image

• US 9,100,561 B2
• Filed: 09/10/2012
• Issued: 08/04/2015
• Est. Priority Date: 09/12/2011
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Komatsu, Satoru
• Primary Examiner(s): Hogue, Dennis
• Application Number:
  US13/607,889
• Publication Number:
  US 20130063652A1
• Time in Patent Office: 1,058 Days
• Field of Search: 348/207.99, 348/207.1, 348/229.1, 348/230.1, 348/239, 348/255, 348362-368, 348/370, 348/371, 600/322, 600/479, 356 3- 22
• US Class Current: 1/1
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 25/53 Control of the integration time

Abstract

An image pickup apparatus includes an image pickup element configured to provide a photoelectric conversion to an optical image formed by an image pickup optical system, a correlation image acquirer configured to obtain a product between a pixel value of an image output from the image pickup element and a representative value of a reference signal that fluctuates during an exposure time period for each of a plurality of divided time periods made by dividing the exposure time period by a predetermined division number, and a correlation image calculator configured to obtain an output from the image pickup element and to calculate a sum of the product for the exposure time period as a correlation image.

Claims

1. An image pickup apparatus comprising:

an image pickup element configured to provide a photoelectric conversion to an optical image formed by an image pickup optical system;

a correlation image acquirer configured to obtain a first product between a pixel value of an image output from the image pickup element and a representative value of a first reference signal that fluctuates during an exposure time period for each of a plurality of divided time periods made by dividing the exposure time period by a predetermined division number such that each of the divided time periods is not more than a half of a period of the first reference signal and to obtain a second product between the pixel value and a representative value of a second reference signal that is a fixed value; and

a correlation image calculator configured to obtain an output from the image pickup element and to calculate a first sum of the first product for the exposure time period as a first correlation image and a second sum of the second product for the exposure time period as a second correlation image.

2. The image pickup apparatus according to claim 1, wherein the correlation image calculator calculates the first sum by dividing the first product by the predetermined division number.

3. The image pickup apparatus according to claim 1, wherein

the correlation image acquirer is configured to obtain a third product between the pixel value and a representative value of a third reference signal that fluctuates during the exposure time period for each of a plurality of divided time periods made by dividing the exposure time period such that each of the divided time periods is not more than a half of a period of the reference signal, the third reference signal being different from the first reference signal;

and

the correlation image calculator is configured to obtain an output from the image pickup element and to calculate a third sum of the third product for the exposure time period as a third correlation image.

4. An image pickup apparatus comprising:

an image pickup element configured to provide a photoelectric conversion to an optical image formed by an image pickup optical system;

a correlation image acquirer configured to obtain a product between a pixel value of an image output from the image pickup element and a representative value of a reference signal that fluctuates during an exposure time period for each of a plurality of divided time periods made by dividing the exposure time period by a predetermined division number;

a correlation image calculator configured to obtain an output from the image pickup element and to calculate a sum of the product for the exposure time period as a correlation image;

a shutter configured to determine an exposure dose upon the image pickup element; and

an exposure controller configured to control opening and closing of the shutter so as to provide a shutter speed modulated by the representative value of the reference signal, and to control the image pickup element so as to provide the photoelectric conversion in synchronization with the opening and closing of the shutter,

wherein the image output from the image pickup element corresponds to the product, and the exposure controller and the shutter serve as the correlation image acquirer.

5. An image pickup apparatus comprising:

an image pickup element configured to provide a photoelectric conversion to an optical image formed by an image pickup optical system;

a correlation image acquirer configured to obtain a product between a pixel value of an image output from the image pickup element and a representative value of a reference signal that fluctuates during an exposure time period for each of a plurality of divided time periods made by dividing the exposure time period by a predetermined division number;

a correlation image calculator configured to obtain an output from the image pickup element and to calculate a sum of the product for the exposure time period as a correlation image;

a light quantity modulator configured to change a light quantity of incident light upon the image pickup element; and

an exposure controller configured to normalize between 0 and 1 a ratio of the light quantity of the incident light that has passed the light quantity modulator, to normalize the reference signal, and to control the light quantity element so as to modulate the ratio of the light quantity of the incident light by a representative value of a normalized reference signal,

wherein the image output from the image pickup element corresponds to the product, and the exposure controller and the light quantity modulator serve as the correlation image acquirer.

6. The image pickup apparatus according to claim 5, wherein the normalized reference signal is a signal of an absolute value of the reference signal in which a maximum value of the signal of the absolute value is 1, and

wherein the correlation image calculator inverts a code of the product corresponding to a minus of the normalized reference signal in calculating the correlation image.

7. An image pickup apparatus comprising:

an image pickup element configured to provide a photoelectric conversion to an optical image formed by an image pickup optical system;

a correlation image acquirer configured to obtain a product between a pixel value of an image output from the image pickup element and a representative value of a reference signal that fluctuates during an exposure time period for each of a plurality of divided time periods made by dividing the exposure time period by a predetermined division number;

a correlation image calculator configured to obtain an output from the image pickup element and to calculate a sum of the product for the exposure time period as a correlation image;

a beam splitter configured to split incident light by the number of types of reference signals;

a plurality of light quantity modulators each corresponding to one of a plurality of image pickup elements corresponding to the plurality of types of reference signals, and each configured to change a light quantity of incident light upon a corresponding one of the image pickup elements; and

an exposure controller configured to normalize between 0 and 1 a ratio of a light quantity of the incident light that has passed the light quantity modulator, to normalize between 0 and 1 a signal of an absolute value of the reference signal, and to control the light quantity modulators so as to modulate the ratio of the light quantity of the incident light by a representative value of a normalized reference signal,

wherein the image output by each image pickup element corresponds to the product and the exposure controller and the light quantity modulators serve as the correlation image acquirer, and

wherein the correlation image calculator inverts a code of the product corresponding to a minus of the normalized reference signal in calculating the correlation image.

8. The image pickup apparatus according to claim 7, wherein the beam splitter splits the incident light by the number of types of reference signals added to 1,

wherein the number of image pickup elements is the number of types of reference signals added to 1, and

wherein the image pickup element which the incident light enters without passing the light quantity modulator outputs the product that is not used to form the correlation image.

9. A method of controlling an image pickup apparatus, the method comprising the steps of:

providing, by an image pickup element, a photoelectric conversion to an optical image formed by an image pickup optical system;

obtaining, by a correlation image acquirer, a first product between a pixel value of an image output from the image pickup element and a representative value of a first reference signal that fluctuates during an exposure time period for each of a plurality of divided time periods made by dividing the exposure time period such that each of the divided time periods is not more than a half of a period of the first reference signal and obtaining a second product between the pixel value and a representative value of a second reference signal that is a fixed value; and

obtaining, by a correlation image calculator, an output from the image pickup element and to calculate a first sum of the first product for the exposure time period as a first correlation image and a second sum of the second product for the exposure time period as a second correlation image.

10. A method of controlling an image pickup apparatus, the method comprising the steps of:

providing, by an image pickup element, a photoelectric conversion to an optical image formed by an image pickup optical system;

obtaining, by a correlation image acquirer, a product between a pixel value of an image output from the image pickup element and a representative value of a reference signal that fluctuates during an exposure time period for each of a plurality of divided time periods made by dividing the exposure time period by a predetermined division number;

obtaining, by a correlation image calculator, an output from the image pickup element and to calculate a sum of the product for the exposure time period as a correlation image;

determining, by a shutter, an exposure dose upon the image pickup element; and

controlling, by an exposure controller, opening and closing of the shutter so as to provide a shutter speed modulated by the representative value of the reference signal, and to control the image pickup element so as to provide the photoelectric conversion in synchronization with the opening and closing of the shutter,

wherein the image output from the image pickup element corresponds to the product, and the exposure controller and the shutter serve as the correlation image acquirer.

11. A method of controlling an image pickup apparatus, the method comprising the steps of:

providing, by an image pickup element, a photoelectric conversion to an optical image formed by an image pickup optical system;

obtaining, by a correlation image acquirer, a product between a pixel value of an image output from the image pickup element and a representative value of a reference signal that fluctuates during an exposure time period for each of a plurality of divided time periods made by dividing the exposure time period by a predetermined division number;

obtaining, by a correlation image calculator, an output from the image pickup element and to calculate a sum of the product for the exposure time period as a correlation image;

changing, by a light quantity modulator, a light quantity of incident light upon the image pickup element; and

normalizing, by an exposure controller, between 0 and 1 a ratio of the light quantity of the incident light that has passed the light quantity modulator, to normalize the reference signal, and to control the light quantity element so as to modulate the ratio of the light quantity of the incident light by a representative value of a normalized reference signal,

wherein the image output from the image pickup element corresponds to the product, and the exposure controller and the light quantity modulator serve as the correlation image acquirer.

12. A non-transitory computer-readable storage medium storing a program which causes a computer to execute a process comprising the steps of:

providing, by an image pickup element, a photoelectric conversion to an optical image formed by an image pickup optical system;

obtaining, by a correlation image acquirer, a first product between a pixel value of an image output from the image pickup element and a representative value of a first reference signal that fluctuates during an exposure time period for each of a plurality of divided time periods made by dividing the exposure time period such that each of the divided time periods is not more than a half of a period of the first reference signal and obtaining a second product between the pixel value and a representative value of a second reference signal that is a fixed value; and

obtaining, by a correlation image calculator, an output from the image pickup element and to calculate a first sum of the first product for the exposure time period as a first correlation image and a second sum of the second product for the exposure time period as a second correlation image.

13. A non-transitory computer-readable storage medium storing a program which causes a computer to execute a process comprising the steps of:

providing, by an image pickup element, a photoelectric conversion to an optical image formed by an image pickup optical system;

obtaining, by a correlation image acquirer, a product between a pixel value of an image output from the image pickup element and a representative value of a reference signal that fluctuates during an exposure time period for each of a plurality of divided time periods made by dividing the exposure time period by a predetermined division number;

obtaining, by a correlation image calculator, an output from the image pickup element and to calculate a sum of the product for the exposure time period as a correlation image;

determining, by a shutter, an exposure dose upon the image pickup element; and

controlling, by an exposure controller, opening and closing of the shutter so as to provide a shutter speed modulated by the representative value of the reference signal, and to control the image pickup element so as to provide the photoelectric conversion in synchronization with the opening and closing of the shutter,

wherein the image output from the image pickup element corresponds to the product, and the exposure controller and the shutter serve as the correlation image acquirer.

14. A non-transitory computer-readable storage medium storing a program which causes a computer to execute a process comprising the steps of:

providing, by an image pickup element, a photoelectric conversion to an optical image formed by an image pickup optical system;

obtaining, by a correlation image acquirer, a product between a pixel value of an image output from the image pickup element and a representative value of a reference signal that fluctuates during an exposure time period for each of a plurality of divided time periods made by dividing the exposure time period by a predetermined division number;

obtaining, by a correlation image calculator, an output from the image pickup element and to calculate a sum of the product for the exposure time period as a correlation image;

changing, by a light quantity modulator, a light quantity of incident light upon the image pickup element; and

normalizing, by an exposure controller, between 0 and 1 a ratio of the light quantity of the incident light that has passed the light quantity modulator, to normalize the reference signal, and to control the light quantity element so as to modulate the ratio of the light quantity of the incident light by a representative value of a normalized reference signal,

wherein the image output from the image pickup element corresponds to the product, and the exposure controller and the light quantity modulator serve as the correlation image acquirer.