Image processing apparatus

• US 6,529,640 B1
• Filed: 06/08/1999
• Issued: 03/04/2003
• Est. Priority Date: 06/09/1998
• Status: Expired due to Term

• Current Assignee: Nikon Corporation
• Original Assignee: Nikon Corporation
• Inventors: Utagawa, Ken, Matsuda, Hideaki
• Primary Examiner(s): Couso, Jose L.
• Assistant Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US09/327,677
• Time in Patent Office: 1,365 Days
• Field of Search: 382/284, 382/300, 382/305, 382/274, 382/254, 382/255, 382/162, 348/64, 348/222.1, 348/234, 348/238, 348/264, 348/340, 348/218, 348/344, 348/348, 348/361, 348/207, 348/229, 348/298, 359/230, 359/227, 396/63, 396/65, 396/89, 396/90, 355/120, 355/79, 355/85
• US Class Current: 382/284
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors

Abstract

The image reading-out apparatus is capable of always obtaining a good image irrespective of a luminance difference. A control unit controls storage times of a pair of image pickup devices through a timing generator independently, thus controlling these storage times to optimized values. Outputs of both of the image pickup devices are subjected to an AD conversion for a digital data processing, and then a synthesis of luminance is performed by a synthesizer. When an automatic mode is selected, a ratio of storage times of both of the image pickup devices is set so that a synthesized image with a suitable dynamic range is obtained in accordance with an object luminance range detected by a luminance range detection section. On the other hand, when a manual mode is selected, the ratio of the storage times of both of the image pickup devices is set so that a synthesized image intended by a photographer depending on which mode is set among a wide dynamic range preference mode, an image quality preference mode and an intermediate mode.

Claims

1. An image taking-in apparatus comprising:

a splitting optical system arranged on an axis incident light from an object, the splitting optical system separating an optical path into two directions;

first and second image pickup means, the first image pickup means being arranged on an optical axis of one optical path separated by said splitting optical system and the second image pickup means being arranged on an optical axis of the other optical path separated by said splitting optical system;

first selecting means for selectively switching between an automatic mode in which the apparatus automatically sets a dynamic range of a picked-up image to a predetermined range and a manual mode in which a photographer manually sets the dynamic range to a desired range;

object luminance range detecting means for detecting an object luminance range that is a luminance range of an object;

AD converting means for performing an AD conversion for outputs of said first and second image pickup means;

synthesizing means for synthesizing the outputs of said first and second image pickup means, which have been subjected to the AD conversion, to output the synthesized output; and

control means for setting a storage time of image information from said first image pickup means and a storage time of image information from said second image pickup means in response to the object luminance range detected by said object luminance range detecting means when the automatic mode is selected by said first selecting means.

2. The image taking-in apparatus according to claim 1, wherein when said first selecting means selects said automatic mode, said control means selects a ratio (T1/T2) of the storage time T1 of said first image pickup means to the storage time T2 of said second image pickup means in response to the object luminance range detected by said object luminance range detecting means.

3. The image taking-in apparatus according to claim 1, further comprising second selecting means which is capable of selectively switching an extent of the dynamic range when a photographer sets the dynamic range of a picked-up image to a desired range manually.

4. The image talking-in apparatus according to claim 3, wherein when the photographer sets the dynamic range of the picked-up image to a desired range manually, a ratio (T1/T2) of the storage time T1 of said first image pickup means to the storage time (T2) of said second image pickup means is selected in accordance with an extent of the dynamic range selected by said second selecting means.

7. The image taking-in apparatus according to any one of claims 1 to 6, wherein when a rate of a light quantity separated into said first image pickup means is indicated by r1 and a rate of a light quantity separated into said second image pickup means is indicated by r2, a ratio (r1/r2) is assumed to be a fixed value other than one.

8. The image taking-in apparatus according to claim 7, wherein when a rate of a light quantity separated into said first image pickup means is indicated by r1 and a rate of a light quantity separated into said second image pickup means is indicated by r2, a ratio (r1/r2) is a fixed value ranging from 2 to 20.

9. The image taking-in apparatus according to claim 8, wherein said ratio (r1/r2) is a fixed value ranging from 4 to 10.

10. The image picking-in apparatus according to claim any one of claims 1 to 6, said apparatus further comprising:

third selecting means which selects a relative relation between a timing of the storage time T1 of said first image pickup means and a timing of the storage time T2 of said second image pickup means at the time of a release operation,

wherein said control means controls said first and second image pickup means in accordance with the relative relation selected by said third selecting means.

11. The image taking-in apparatus according to claim 10, wherein when a difference between completion times at which the storage of said first and second image pickup devices terminate is equal to a predetermined value or more, image information outputted from one image pickup means with a shorter storage time is first read out, thus storing the read-out image information in memory means temporarily, and image information outputted from the other image pickup means with a longer storage time is subsequently read out, thus synthesizing images by the synthesizing means based on the image information stored in the memory means and the image information of the other image pickup means.

12. The image taking-in apparatus according to any one of claims 1 to 6, further comprising:

processing means for transforming a grayscale of each of image information outputted from said first and second image pickup means,

wherein an image synthesis is performed by the synthesizing means based on the image information that has been subjected to the grayscale transformation by said processing means.

13. The image taking-in apparatus according to any one of claims 1 and 5, wherein said object luminance range detecting means is realized by using said first and second image pickup devices, and said storage times T1 of said first image pickup device and T2 of said second image pickup device are changed in a state where a ratio (T1/T2) of the storage time T1 to the storage time T2 is kept at a predetermined value or more, thereby detecting an object luminance range.

14. The image taking-in apparatus according to any one of claims 1 to 6, wherein said synthesizing means performs the synthesis at a luminance overlapping portion where a luminance of image information outputted from said first and second image pickup devices by using a weight function which value changes in accordance with a luminance continuously, and said weight function changes its shape and parameter in accordance with any one of an extension of said object luminance range and an extension of the dynamic range which is set manually by the photographer.

15. The image taking-in apparatus according to any one of claims 1 to 6, wherein said synthesizing means performs the synthesis by forcibly adding the image informations outputted by said first and second image pickup means.

16. The image taking-in apparatus according to any one of claims 1 to 6, further comprising:

a strobe;

fourth selecting means for selecting a flash mode of the strobe; and

strobe reaching region discriminating means for detecting a region where the strobe reaches in the image,

wherein said synthesizing means performs a synthesis at a luminance overlapping portions of image informations outputted from said first and second image pickup devices by using a weight function which value changes in accordance with a luminance continuously, and the weight function changes its shape and parameter in accordance with results discriminated by said strobe reaching region discriminating means.

17. The image taking-in apparatus according to any one of claims 1 to 6, further comprising:

a strobe; and

fourth selecting means for selecting a flash mode of the strobe,

wherein said control means controls said first and second image pickup devices so that storage times of said first and second image pickup devices become equal to each other automatically at the time of use of the strobe, and said synthesizing means performs a synthesis by forcibly adding image information outputted from said first and second image pickup devices.

18. The image taking-in apparatus according to any one of claims 1 to 6, wherein said first and second image pickup devices are color image pickup means each capable of taking in a full color image.

19. The image taking-in apparatus according to claim 18, wherein said color image pickup means has an on-chip color filter, pixels are arranged in the relatively same position, and colors of the on-chip color filter are arranged in relatively shifted positions.

20. The image taking-in apparatus according to any one of claims 1 to 6, wherein in accordance with one of an extension of said object luminance range and an extension of the dynamic range which is set by the photographer, an algorithm of the synthesis by said synthesizing means is switched.

21. The image taking-in apparatus according to any one of claims 1 to 6, wherein said first and second image pickup means are color image pickup means each capable of taking a full color image, when said object luminance range is narrow, or when an extension of the dynamic range which is set by the photographer manually is narrow, said synthesizing means synthesizes images using an algorithm which prefers a prevention of a false color and an enhancement of a resolution, and when said object luminance range is broad, or when an extension of the dynamic range which is set by the photographer manually is broad, said synthesizing means synthesizes images by using an algorithm which prefers a dynamic range extension.

22. The image taking-in apparatus according to claim 10, further comprising:

a strobe; and

fourth selecting means for selecting a flash mode of the strobe,

wherein when the strobe is used, timing of the flash of the strobe is allowed to match with the timing selected by said third selecting means, and the strobe is made to flash in accordance with the exposing times of said first and second image pickup means.

5. An image taking-in apparatus comprising:

a splitting optical system arranged on an axis of incident light from an object, the splitting optical system separating an optical path into two directions and keeping a light quantity separation ratio at a predetermined value;

first and second image pickup means, the first image pickup means being arranged on an optical axis of one optical path separated by said splitting optical system and the second image pickup means being arranged on an optical axis of the other optical path separated by said splitting optical system;

object luminance range detecting means for detecting an object luminance range that is a luminance range of an object;

AD converting means for performing an AD conversion for outputs of said first and second image pickup means;

synthesizing means for luminance synthesizing outputs of said first and second image pickup means, which have been subjected to the AD conversion, to output a synthesized output; and

control means for setting a storage time of image information from said first image pickup means and a storage time of image information from said second image pickup means in response to the object luminance range detected by said object luminance range detecting means.

6. An image taking-in apparatus comprising:

a splitting optical system arranged on an axis of incident light from an object, the splitting optical system separating an optical path into two directions;

first and second image pickup means, the first image pickup means being arranged on an optical axis of one optical path split by said splitting optical system and second image pickup means being arranged on an optical axis of the other optical path split by said splitting optical system;

adjusting means provided so that a photographer manually sets a dynamic range of a photographed image to a desired range;

AD converting means for performing an AD conversion for outputs of said first and second image pickup means;

synthesizing means for luminance synthesizing the outputs of said first and second image pickup means, which have been subjected to the AD conversion, to output the synthesized output; and

control means for setting a storage time of image information from said first image pickup means and a storage time of image information from said second image pickup means in response to said desired range set by said adjusting means.

23. An image taking-in apparatus comprising:

a splitting optical system arranged on an axis of incident light from an object, the splitting optical system separating an optical path into two directions;

first and second image pickup means in which color filters with a Bayer array are provided, the first image pickup means being arranged on an optical axis of one optical path split by said splitting optical system and the second image pickup means being arranged on an optical axis of the other optical path split by said splitting optical system, both of image pickup means being relatively offset by one pixel from each other;

AD converting means for performing an AD conversion for outputs of said first and second image pickup means;

synthesizing means for luminance synthesizing the outputs of said first and second image pickup means, which have been subjected to the AD conversion, to output the synthesized output; and

control means for setting a storage time of image information from said first image pickup means and a storage time of image information from said second image pickup means in response to an object luminance range measured or a dynamic range of a photographed image which is set by a photographer manually.