Image capturing apparatus, method, and program for performing an auto focus operation using invisible and visible light

• US 8,537,264 B2
• Filed: 04/28/2008
• Issued: 09/17/2013
• Est. Priority Date: 05/01/2007
• Status: Active Grant

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ono, Hiroaki
• Primary Examiner(s): SPINKS, ANTOINETTE T
• Application Number:
  US12/110,770
• Publication Number:
  US 20080273099A1
• Time in Patent Office: 1,968 Days
• Field of Search: 348/349, 348/335, 348/345, 348/353
• US Class Current: 348/345
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 23/675 comprising setting of focusing regions

Abstract

An image capturing apparatus includes an image pickup unit including an image pickup element that captures a plurality of colors in ranges of visible and invisible light incoming through an image capturing lens, a pixel value calculating unit that calculates a pixel value of each pixel regarding at least one of the captured colors, a contrast value calculating unit that calculates a contrast value for each color based on the calculated pixel values, a correction value calculating unit that calculates a focus correction value for a distance from a current position of the image capturing lens to a focus position based on a temporal change in the calculated contrast values of the at least one color, and an image capturing lens driving unit that drives the image capturing lens based on the calculated focus correction value.

Claims

1. An image capturing apparatus comprising:

image pickup means including an image pickup element configured to capture, in a unit of pixels, a plurality of colors in ranges of visible and invisible light incoming through an image capturing lens;

pixel value calculating means configured to calculate a pixel value of each pixel regarding at least one of the colors captured by the image pickup means;

contrast value calculating means configured to calculate a contrast value for each of the at least one color on the basis of the pixel values calculated by the pixel value calculating means;

contrast value storage memory means configured to store a plurality of contrast values;

focus correction value calculating means configured to determine whether the contrast values have a relative maximum value and calculate a focus correction value for a distance from a current position of the image capturing lens to a focus position of the image capturing lens on the basis of a temporal change in the contrast values of the at least one color calculated by the contrast value calculating means; and

image capturing lens driving means configured to drive the image capturing lens on the basis of the focus correction value calculated by the focus correction value calculating means;

wherein the pixel value calculating means calculates, for each pixel, a luminance value, including the visible light and the invisible light, as a pixel value;

wherein the contrast value calculating means calculates the contrast value by applying a high-pass filter to the luminance value of each pixel calculated by the pixel value calculating means;

wherein when the focus correction value calculating means determines that the contrast values do not have a relative maximum value, the focus correction value calculating means calculates the focus correction value for the distance from the current position of the image capturing lens to the focus position of the image capturing lens on the basis of the temporal change in the contrast values calculated by the contrast value calculating means;

wherein the focus correction value calculating means determines whether the contrast values have a relative maximum value on the basis of a contrast value calculated for incoming colors at a current time, a plurality of contrast values calculated for incoming colors before the current time and stored in the contrast value storage memory means, and a corresponding position of the image capturing lens;

wherein at least three contrast values are needed for the contrast values to have a relative maximum value; and

wherein when two or more contrast values exist and the focus correction value calculating means determines that the contrast values do not have a relative maximum value, the focus calculating means sets, as the focus correction value, a distance between the current position of the image capturing lens and a position away from the current position of the image capturing lens corresponding to a maximum value of the plurality of the contrast values calculated before the current time by a predetermined distance in a direction from the current position of image capturing lens toward the position corresponding to the maximum value.

2. The apparatus according to claim 1, wherein

the pixel value calculating means includes

visible light value calculating means configured to calculate, for each pixel, a luminance value of visible light as a visible light value, and

invisible light value calculating means configured to calculate, for each pixel, a luminance value of invisible light as an invisible light value, and

wherein the contrast value calculating means includes

visible light contrast calculating means configured to calculate a visible light contrast value regarding the visible light values calculated by the visible light value calculating means, and

invisible light contrast calculating means configured to calculate an invisible light contrast value regarding the invisible light values calculated by the invisible light value calculating means, and wherein the focus correction value calculating means calculates the focus correction value for the distance from the current position of the image capturing lens to the focus position of the image capturing lens on the basis of the temporal change in the visible light contrast values and the invisible light contrast values.

3. The apparatus according to claim 2, further comprising:

visible light S/N ratio calculating means configured to calculate an S/N ratio of the visible light contrast value calculated by the visible light contrast value calculating means; and

invisible light S/N ratio calculating means configured to calculate an S/N ratio of the invisible light contrast value calculated by the invisible light contrast value calculating means, wherein

the focus correction value calculating means compares the S/N ratio of the visible light contrast value with the S/N ratio of the invisible light contrast value, selects the contrast value having a larger SIN ratio as a contrast value suitable for calculation of the focus correction value, and calculates the focus correction value for the distance from the current position of the image capturing lens to the focus position of the image capturing lens on the basis of a temporal change in the contrast values suitable for calculation of the focus correction value.

4. The apparatus according to claim 2, further comprising:

contrast value combining means configured to combine the visible light contrast value and the invisible light contrast value to generate a combined contrast value, wherein

the focus correction value calculating means calculates the focus correction value for the distance from the current position of the image capturing lens to the focus position of the image capturing lens on the basis of a temporal change in the combined contrast values.

5. The apparatus according to claim 4, wherein the contrast value combining means combines the visible light contrast value and the invisible light contrast value at a combination ratio based on both of the contrast values to generate the combined contrast value, and

wherein the focus correction value calculating means calculates, according to the combination ratio, the focus correction value for the distance from the current position of the image capturing lens to the focus position of the image capturing lens on the basis of the temporal change in the combined contrast values.

6. The apparatus according to claim 1, further comprising:

auxiliary light irradiating means configured to irradiate light of a color, in the visible and invisible light ranges, suitable for calculation of the contrast value onto a subject; and

auxiliary light irradiation control means configured to control irradiation of auxiliary light performed by the auxiliary light irradiating means.

7. The apparatus according to claim 6, wherein the auxiliary light irradiation control means controls the auxiliary light irradiating means to irradiate light, selected from the visible light, the infrared light, and the ultraviolet light, suitable for calculation of the contrast value onto a subject.

12. The apparatus according to claim 1, wherein when the at least three contrast values exist and the relative maximum contrast value exists when the position of the image capturing lens is located near a middle of a position nearest to an aperture stop and farthest from an aperture stop, the focus correction value calculating means determines a position of the image capturing lens that gives the relative maximum contrast value via interpolation using the contrast value corresponding to the middle position of the image capturing lens and calculates a difference between the middle position and a current position as the focus correction value.

8. An image capturing method comprising the steps of:

capturing, in a unit of pixels, a plurality of colors in ranges of visible and invisible light incoming through an image capturing lens;

calculating a pixel value of each pixel regarding at least one of the captured colors;

calculating, for each of the at least one color, a contrast value on the basis of the calculated pixel values of the corresponding color;

storing a plurality of contrast values;

determining whether the contrast values have a relative maximum value; and

calculating a focus correction value for a distance from a current position of the image capturing lens to a focus position of the image capturing lens on the basis of a temporal change in the calculated contrast values of the at least one color; and

driving the image capturing lens on the basis of the calculated focus correction value;

wherein calculating a pixel value includes calculating, for each pixel, a luminance value, including the visible light and the invisible light, as a pixel value;

wherein calculating a contrast value includes calculating the contrast value by applying a high-pass filter to the luminance value of each pixel;

wherein when the determination is made that the contrast values do not have a relative maximum value, calculating the focus correction value for the distance from the current position of the image capturing lens to the focus position of the image capturing lens on the basis of the temporal change in the contrast;

wherein determining whether the contrast values have a relative maximum value is based on a contrast value calculated for incoming colors at a current time, a plurality of contrast values calculated for incoming colors stored before the current time, and a corresponding position of the image capturing lens;

wherein at least three contrast values are needed for the contrast values to have a relative maximum value; and

wherein when two or more contrast values exist and the contrast values do not have a relative maximum setting, as the focus correction value, a distance , between the current position of the image capturing lens and a position away from the current position of the image capturing lens corresponding to a maximum value of the plurality of contrast values , calculated before the current time by a predetermined distance in a direction from the current position of image capturing lens toward the position corresponding to the maximum value.

9. A non-transitory program-storing medium having a program stored thereon, the program allowing a computer to execute a method, the method comprising the steps of:

capturing, in a unit of pixels, a plurality of colors in ranges of visible and invisible light incoming through an image capturing lens;

calculating a pixel value of each pixel regarding at least one of the captured colors;

calculating, for each of the at least one color, a contrast value on the basis of the calculated pixel values of the corresponding color;

storing a plurality of contrast values;

determining whether the contrast values have a relative maximum value; and

calculating a focus correction value for a distance from a current position of the image capturing lens to a focus position of the image capturing lens on the basis of a temporal change in the calculated contrast values of the at least one color; and

driving the image capturing lens on the basis of the calculated focus correction value;

wherein calculating a pixel value includes calculating, for each pixel, a luminance value, including the visible light and the invisible light, as a pixel value;

wherein calculating a contrast value includes calculating the contrast value by applying a high-pass filter to the luminance value of each pixel;

wherein when the determination is made that the contrast values do not have a relative maximum value, calculating the focus correction value for the distance from the current position of the image capturing lens to the focus position of the image capturing lens on the basis of the temporal change in the contrast values;

wherein determining whether the contrast values have a relative maximum value is based on a contrast value calculated for incoming colors at a current time, a plurality of contrast values calculated for incoming colors stored before the current time, and a corresponding position of the image capturing lens;

wherein at least three contrast values are needed for the contrast values to have a relative maximum value; and

wherein when two or more contrast values exist and the contrast values do not have a relative maximum value, setting, as the focus correction value, a distance between the current position of the image capturing lens and a position away of the current position of the image corresponding to a maximum value of the plurality of contrast values calculated before the current time by a predetermined distance in a direction from the current position of capturing lens toward the position corresponding to the maximum value.

10. An image capturing apparatus comprising:

an image pickup unit including an image pickup element configured to capture, in a unit of pixels, a plurality of colors in ranges of visible and invisible light incoming through an image capturing lens;

a pixel value calculating unit configured to calculate a pixel value of each pixel regarding at least one of the colors captured by the image pickup unit;

a contrast value calculating unit configured to calculate a contrast value for each of the at least one color on the basis of the pixel values calculated by the pixel value calculating unit;

contrast value storage memory unit configured to store a plurality of contrast values;

a focus correction value calculating unit configured to determine whether the contrast values have a relative maximum value and calculate a focus correction value for a distance from a current position of the image capturing lens to a focus position of the image capturing lens on the basis of a temporal change in the contrast values of the at least one color calculated by the contrast value calculating unit; and

an image capturing lens driving unit configured to drive the image capturing lens on the basis of the focus correction value calculated by the focus correction value calculating unit;

wherein the pixel value calculating unit calculates, for each pixel, a luminance value, including the visible light and the invisible light, as a pixel value;

wherein the contrast value calculating unit calculates the contrast value by applying a high-pass filter to the luminance value of each pixel calculated by the pixel value calculating unit;

wherein when the focus correction value calculating unit determines that the contrast values do not have a relative maximum value, the focus correction value calculating unit calculates the focus correction value for the distance from the current position of the image capturing lens to the focus position of the image capturing lens on the basis of the temporal change in the contrast values calculated by the contrast value calculating unit;

wherein the focus correction value calculating unit determines whether the contrast values have a relative maximum value on the basis of a contrast value calculated for incoming colors at a current time, a plurality of contrast values calculated for incoming colors before the current time and stored in the contrast value storage memory unit, and a corresponding position of the image capturing lens;

wherein at least three contrast values are needed for the contrast values to have a relative maximum value,

wherein when two or more contrast values exist and the focus correction value calculating means determines that the contrast values do not have a relative maximum value, the focus correction value calculating means sets, as the focus correction value, a distance between the current position of the image capturing lens and a position away from the current position of the image capturing lens corresponding to a maximum value of the plurality of contrast values calculated before the current time by a predetermined distance in a direction from the current position of image lens toward the position corresponding to the maximum value.

11. An image capturing apparatus comprising:

image pickup means including an image pickup element configured to capture, in a unit of pixels, a plurality of colors in ranges of visible and invisible light incoming through an image capturing lens;

pixel value calculating means configured to calculate a pixel value of each pixel regarding at least one of the colors captured by the image pickup means;

contrast value calculating means configured to calculate a contrast value for each of the at least one color on the basis of the pixel values calculated by the pixel value calculating means;

contrast value storage memory means configured to store a plurality of contrast values;

focus correction value calculating means configured to determine whether the contrast values have a relative maximum value and calculate a focus correction value for a distance from a current position of the image capturing lens to a focus position of the image capturing lens on the basis of a temporal change in the contrast values of the at least one color calculated by the contrast value calculating means; and

image capturing lens driving means configured to drive the image capturing lens on the basis of the focus correction value calculated by the focus correction value calculating means;

wherein the pixel value calculating means calculates, for each pixel, a luminance value of infrared light and a luminance value of ultraviolet light as an infrared light value and an ultraviolet light value, respectively;

wherein the contrast value calculating means calculates a contrast value by applying a high-pass filter to each of the visible light values, the infrared light values, and the ultraviolet light values calculated by the pixel value calculating means;

wherein when the focus correction value calculating means determines that the contrast values do not have a relative maximum value, the focus correction value calculating means calculates the focus correction value for the distance from the current position of the image capturing lens to the focus position of the image capturing lens on the basis of temporal changes in the contrast values of the visible light values, the infrared light values, and the ultraviolet light values calculated by the contrast value calculating means;

wherein the focus correction value calculating means determines whether the contrast values have a relative maximum value on the basis of a contrast value calculated for incoming colors at a current time, a plurality of contrast values calculated for incoming colors before the current time and stored in the contrast value storage memory means, and a corresponding position of the image capturing lens;

wherein at least three contrast values are needed for the contrast values to have a relative maximum value; and

wherein when two or more contrast values exist and the focus correction value calculating means determines that the contrast values do not have a relative maximum value, the focus correction value calculating means sets, as the focus correction value, a distance between the current position of the image capturing lens and a position away from the current position of the image capturing lens corresponding to a maximum value of the plurality of contrast values calculated before the current time by a predetermined distance in a direction from the current position of image capturing lens toward the position corresponding to the maximum value.