Digital cameras using multiple sensors with multiple lenses

• US 6,611,289 B1
• Filed: 01/15/1999
• Issued: 08/26/2003
• Est. Priority Date: 01/15/1999
• Status: Expired due to Term

• Current Assignee: Yanbin Yu, Zhongxuan Zhang
• Original Assignee: Yanbin Yu, Zhongxuan Zhang
• Inventors: Yu, Yanbin, Zhang, Zhongxuan
• Primary Examiner(s): Garber, Wendy R.
• Assistant Examiner(s): TOPPIN, CATHERINE J
• Application Number:
  US09/232,769
• Time in Patent Office: 1,684 Days
• Field of Search: H04/N.9090, H04/N.9093, 348/272, 348/279, 348/265
• US Class Current: 348/265
• CPC Class Codes: H04N 2209/048 having several pick-up sensors
• CPC Class Codes: H04N 23/15 Image signal generation with circuitry for avoiding or correcting image misregistration

Abstract

An improved digital camera that produces digital images of high qualities without using expensive image sensors and optics is disclosed. The disclosed digital cameras use multiple image sensors with multiple lenses. One of the multiple image sensors is made to be responsive to all intensity information in visible color spectrum and a (gray intensity) image resulting from the sensor is used to compensate lost information in images from other image sensors responsive to certain colors. A final color image is obtained by a digital image processing circuitry that performs pixel registration process with reference to the gray intensity image so that a true color image with true resolution is obtained therefrom.

Claims

1. An improved digital camera comprising:

a first and a second image sensor closely positioned with respect to a common plane, said second image sensor sensitive to a full region of visible color spectrum;

two lenses, each being mounted in front of one of said two image sensors;

said first image sensor producing a first image and said second image sensor producing a second image;

an analog-to-digital converting circuitry coupled to said first and said second image sensor and digitizing said first and said second intensity images to produce correspondingly a first digital image and a second digital image;

an image memory, coupled to said analog-to-digital converting circuitry, for storing said first digital image and said second digital image; and

a digital image processor, coupled to said image memory and receiving said first digital image and said second digital image, producing a resultant digital image from said first digital image enhanced with said second digital image.

2. The improved digital camera as recited in claim 1, wherein said first image sensor sensitive to said full region of visible color spectrum.

3. The improved digital camera as recited in claim 1, wherein one of said two lenses is configured to allow said first image sensor sensitive to a selected range of said full region of visible color spectrum, and the other one of said two lenses is configured to allow said second image sensors sensitive to said full region of visible color spectrum.

4. The improved digital camera as recited in claim 1, wherein said analog-to-digital converting circuitry comprises two individual analog-to-digital converters, each integrated with one of said first and second image sensors so that said first and second digital images are digitized independently and in parallel to increase signal throughput rate.

5. The improved digital camera as recited in claim 1, wherein said digital image processor increases the dynamic range of said first digital image by incorporating a portion of said second digital image into said first digital image.

6. An improved digital camera comprising;

four image sensors, closely positioned with respect to a common plane,

four lenses, each mounted in front of one of the four image sensors;

first three of said four image sensors being respectively sensitive to three different regions of visible color spectrum, a fourth sensor of said four image sensors being sensitive to a full region of said visible color spectrum, said full region including said three different regions in said visible color spectrum;

said four image sensors producing, independently and respectively, four intensity images when being exposed to an imaging target, first three images of said four intensity images respectively from said first three of said four image sensors and a fourth image of said four intensity images from said fourth sensor of said four image sensors;

an analog-to-digital converting circuitry coupled to said four image sensors and digitizing said four intensity images to produce four digital images, first three of said four digital images corresponding to said first three images and a fourth digital image of said four digital images corresponding to said fourth image of said four intensity images;

an image memory, coupled to said analog-to-digital converting circuitry, for storing said four digital images; and

a digital image processing circuitry coupled to said image memory and receiving said four digital images, producing a color image of said imaging target from said four digital images.

7. The improved digital camera as recited in claim 6, wherein said first three of said four image sensors, each being coated with a different color filter responsive to one of said three different regions in said visible color spectrum.

8. The improved digital camera as recited in claim 7, wherein said different color filter is in one of the three primary colors and wherein said three different regions are respectively the three primary colors.

9. The improved digital camera as recited in claim 8, wherein the three primary colors are red, green and blue.

10. The improved digital camera as recited in claim 7, wherein said different color filter is in one of the three complementary colors and wherein said three different regions are respectively the three complementary colors.

11. The improved digital camera as recited in claim 10, wherein the three complementary colors are magenta, yellow and cyan.

12. The improved digital camera as recited in claim 6, wherein three of said four lenses are respectively in three primary colors.

13. The improved digital camera as recited in claim 12, wherein the three primary colors are red, green and blue.

14. The improved digital camera as recited in claim 6, wherein three of said four lenses are respectively in three complementary colors.

15. The improved digital camera as recited in claim 14, wherein the three complementary colors are magenta, yellow and cyan.

16. The improved digital camera as recited in claim 6, wherein said analog-to-digital converting circuitry comprising four independent analog-to-digital converters, each integrated with one of said four image sensors so that said four intensity images are digitized independently and in parallel to increase signal throughput rates.

17. The improved digital camera as recited in claim 6, wherein said digital image processing circuitry increases respectively and independently the dynamic range of said first three digital images by incorporating a portion of said fourth digital image into each of said first three digital images.

18. The improved digital camera as recited in claim 6, wherein said digital image processing circuitry produces four control signals, each controlling the exposure time of one of said four image sensors by analyzing respectively said four digital images.

19. The improved digital camera as recited in claim 18, wherein said digital image processing circuitry generating a histogram for each of said four digital images, each of said controls signals calculated from said histogram.

20. The improved digital camera as recited in claim 6, wherein said four image sensors are monochromatic and identical.

21. The improved digital camera as recited in claim 20, wherein said digital image processing circuitry calculates a set of statistic measurement within a moving window for each of said four digital images.

22. The improved digital camera as recited in claim 20, wherein said digital image processing circuitry further minimizes the difference of said statistic measurements between said fourth digital image and each of said first three digital images by moving center coordinates of said moving window in each of said first three digital images towards corresponding center coordinates of said moving window in said fourth digital image.

23. The improved digital camera as recited in claim 22, wherein said digital image processing circuitry further derives a motion vector when the difference of said statistic measurements between said fourth digital image and each of said first three digital images is minimized so that the spatial position offset among said four image sensors can be determined.

24. The improved digital camera as recited in claim 23, wherein said digital image processing circuitry registers each pixel in each of said first three digital images with reference to corresponding pixels in said fourth digital image.

25. The improved digital camera as recited in claim 24, wherein said digital image processing circuitry enhances each of said first three digital images by looking up said fourth digital image.

26. A method for generating color images of high quality in a digital camera, said method comprising:

obtaining three scalar images from three image sensors closely positioned in a common plane with reference to an image target;

obtaining a gray intensity image from a fourth image sensor, said fourth image sensor closely positioned in said common plane with said three image sensors;

digitizing said three scalar intensity images and said gray intensity image to produce three scalar digital images and a gray digital image;

buffering said three scalar digital images and said gray digital image in an image memory;

producing a color Image from said three scalar digital images processed in conjunction with said gray digital image; and

wherein said three image sensors and said fourth image sensors are monochromatic and identical in resolution.

27. The method as recited in claim 26, wherein said three image sensors are made to be respectively responsive to three primary colors.

28. The method as recited in claim 27, wherein said three image sensors are respectively coated with filters of the three primary colors.

29. The method as recited in claim 27, wherein each of said three image sensors is mounted behind a lens.

30. The method as recited in claim 26, wherein said digitizing said three scalar intensity images and said gray intensity image comprises:

digitizing said three scalar intensity images and said gray intensity image in parallel and simultaneously by four individual A/D converts respectively integrated with said three image sensors and said fourth image sensor.

31. The method as recited in claim 26, wherein said producing a color image from said three scalar digital images processed in conjunction with said gray digital image comprises:

determining a set of reference statistic measurement in a windowed area of said gray digital image, said windowed area having referenced center coordinates;

determining a set of statistic measurement in a corresponding windowed area of each of said three scalar digital images;

minimizing the difference between said reference statistic measurement and said statistic measurement by moving center coordinates of said windowed area of each of said three scalar digital images towards said referenced center coordinates;

obtaining a motion vector to determine a spatial position offset between said gray digital image and each of three scalar digital images;

registering each pixels of three scalar digital images using said motion vector.