One chip, low light level color camera

• US 7,012,643 B2
• Filed: 05/08/2002
• Issued: 03/14/2006
• Est. Priority Date: 05/08/2002
• Status: Expired due to Term

• Current Assignee: Ball Aerospace & Technologies Corporation (Ball Corporation)
• Original Assignee: Ball Aerospace & Technologies Corporation (Ball Corporation)
• Inventors: Frame, Wayne W.
• Primary Examiner(s): Ometz, David L.
• Assistant Examiner(s): AGGARWAL, YOGESH K
• Application Number:
  US10/142,320
• Publication Number:
  US 20030210332A1
• Time in Patent Office: 1,406 Days
• Field of Search: 348/216.1, 348/217.1, 348/235, 348/236, 348/238, 348/164, 348272-274, 348/275, 348/276, 348/277, 250/331, 250/339.05, 250/339.01, 250/339.02
• US Class Current: 348/272
• CPC Class Codes: H04N 2209/045 using mosaic colour filter
• CPC Class Codes: H04N 2209/047 using multispectral pick-up elements
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements

Abstract

A high sensitivity, single chip, low light level imaging device is provided. The imaging device of the present invention utilizes sparse color sampling, to maximize the luminance information gathered. In particular, color information is gathered by a small proportion of the pixels included in an image sensor, while the remaining pixels operate at full spectrum sensitivity. The present invention allows the correct hue of objects to be determined, while providing high sensitivity to available luminance information in a scene. In addition, the present invention allows hue information to be ascribed to objects in a scene, even in combination with luminance data collected over a spectrum that includes the near infrared and infrared wavelengths.

Claims

1. A method for providing a low light level camera, comprising:

providing an image sensor comprising a plurality of pixels located within a first area to define a single focal surface;

filtering image information to a first proportion of said pixels for sensitivity to within a selected first spectral region; and

filtering image information to a second proportion of said pixels for sensitivity to within a selected second spectral region, wherein less than one-third of said image sensor pixels receive image information filtered to within said first or second spectral regions, wherein more than one-third of said pixels of said image sensor receive image information that is not filtered to within said first and second selected spectral regions, and wherein said more than one-third of said pixels are sensitive to light having a wavelength greater than about 780 nanometers.

2. The method of claim 1, wherein said image information that is not filtered to within said first or second spectral regions comprises a luminance signal, and wherein said filtered image information comprises a chrominance signal.

3. The method of claim 1, wherein said first proportion of pixels and said second proportion of pixels are pseudo-randomly distributed among all of said pixels of said image sensor.

4. The method of claim 1, wherein said first proportion of pixels and said second proportion of pixels are distributed about lines that are not parallel to sides of said image sensor.

5. The method of claim 2, further comprising outputting an image sensed by said low light level camera, wherein at least a first part of said image is derived from an area of said image sensor comprising a pixel receiving image information that is not filtered to within said first or second spectral regions and wherein chrominance information for said first part of said image is derived from at least a first pixel receiving filtered image information located at least one of within and proximate to said at least a first part of said image sensor.

6. The method of claim 2, further comprising outputting an image sensed by said low light level camera, wherein at least a first part of said image comprising a pixel receiving unfiltered image information is provided with chrominance information derived from a plurality of pixels receiving filtered image information.

7. The method of claim 2, further comprising:

adjusting a gain associated with said chrominance signal.

8. The method of claim 7, wherein said gain is reduced in response to a decrease in a signal to noise ratio of said chrominance signal.

9. The method of claim 2, further comprising:

delaying at least one of a frame and a field of said chrominance signal relative to a corresponding at least one of a frame and a field of said luminance signal; and

performing a signal processing operation with respect to said chrominance signal.

10. The method of claim 2, further comprising:

averaging a plurality of at least one of frames and fields of said chrominance signal.

11. The method of claim 2, wherein said image sensor comprises a first set of pixels comprising a first frame and a second set of pixels comprising a second frame, said method further comprising:

generating said chrominance signal with respect to said first frame but not said second frame.

12. The method of claim 2, wherein in a first mode of operation said chrominance signal is processed and displayed and wherein in a second mode of operation said chrominance signal is not processed and is not displayed.

13. The method of claim 1, wherein 15% or less of said image sensor pixels receive image information filtered to have sensitivity within said first or second spectral regions.

14. The method of claim 1, wherein 12% or less of said image sensor pixels receive image information filtered to have sensitivity within said first or second spectral regions.

15. The method of claim 1, further comprising providing said first and second proportions of pixels and pixels not included in said first and second proportions of pixels image information simultaneously.

16. The method of claim 1, wherein said single focal surface comprises a focal plane.

17. The method of claim 1, wherein no more than 10% of said image sensor pixels receive image information filtered for sensitivity to within one of said first and second spectral regions.

18. A method for providing a low light level camera, comprising:

providing an image sensor comprising a plurality of pixels located within a first area to define a single focal surface;

filtering image information to a first proportion of said pixels for sensitivity to within a selected first spectral region;

filtering image information to a second proportion of said pixels for sensitivity to within a selected second spectral region; and

filtering image information to a third proportion of said pixels for sensitivity to within a selected third spectral region, wherein less than one-third of said image sensor pixels receive image information filtered for sensitivity to within said first, second and third spectral regions.

19. The method of claim 18, wherein said first proportion of pixels, said second proportion of pixels and said third proportion of pixels are randomly distributed among all of said pixels of said image sensor.

20. The method of claim 18, wherein a plurality of elements comprising four filter units for four of said image sensor pixels are formed, wherein for each of said elements first and second of said filter units pass energy in said first spectral region, a third of said filter units passes energy in said second spectral region, and a fourth of said filter units passes energy in said third spectral region.

21. The method of claim 20, wherein said elements are distributed randomly among said pixels of said image sensor.

22. The method of claim 20, wherein said elements are distributed about lines that are not parallel to edges of said image sensor.

23. The method of claim 18, wherein said image information provided to said pixels not included in said first, second and third proportions of pixels comprises full spectrum image information.

24. The method of claim 18, wherein 15% or less of said image sensor pixels receive image information filtered to within one of said first, second or third spectral regions.

25. The method of claim 18, wherein 12% or less of said image sensor pixels receive image information filtered to within one of said first, second or third spectral regions.

26. The method of claim 18, wherein pixels receiving said filtered image information are arranged in a plurality of groups, and wherein most of said groups are surrounded by a plurality of pixels that do not receive image information filtered to within one of said first, second and third spectral regions.

27. The method of claim 18, wherein no more than 10% of said image sensor pixels receive image information filtered for sensitivity to within one of said first, second and third spectral regions.

28. A method for providing a low light level camera, comprising:

providing an image sensor comprising a plurality of pixels located within a first area to define a single focal surface;

filtering image information to a first proportion of said pixels for sensitivity to within a selected first spectral region; and

filtering image information to a second proportion of said pixels for sensitivity to within a selected second spectral region, wherein less than one-third of said image sensor pixels receive image information filtered to within said first or second spectral regions, wherein said first and second spectral regions overlap and no more than 10% of said image sensor pixels receive image information filtered for sensitivity to within one of said first and second spectral regions.

29. A device for color imaging in low light level conditions, comprising:

an image sensor, wherein said image sensor comprises;

a first plurality of pixels particularly sensitive to luminance;

a second plurality of pixels particularly sensitive to chrominance, wherein said first plurality of pixels comprises more than two-thirds of a total number of pixels of said image sensor, wherein at least some pixels included in said first plurality pixels are adjacent to at least one of said pixels included in said second plurality of pixels, wherein said second plurality of pixels are arranged in groups of three, and wherein each of said groups comprises a first pixel having sensitivity to a first spectral band, a second pixel having sensitivity to a second spectral band, and a third pixel having sensitivity to a third spectral band.

30. The device of claim 29, wherein said second plurality of pixels are weighted about lines that are not parallel to edges of said image sensor.

31. The device of claim 29, wherein said second plurality of pixels are distributed about diagonals of said image sensor.

32. The device of claim 29, wherein pixels included in said second plurality of pixels are distributed among said first plurality of pixels such that each pixel included in said second plurality of pixels is adjacent to at least one pixel included in said first plurality of pixels.

33. The device of claim 29, wherein each of said three pixels is sensitive to a different range of wavelengths, and wherein at least two edges of each of said groups are bordered by pixels included in said first plurality of pixels.

34. The device of claim 29, wherein at least most of said groups of pixels included in said second plurality of pixels are surrounded by a number of pixels included in said first plurality of pixels.

35. The device of claim 29, wherein said second plurality of pixels comprises less than 10% of said total number of pixels of said image sensor.

36. A device for color imaging in low light level conditions, comprising:

an image sensor, wherein said image sensor comprises;

a first plurality of pixels particularly sensitive to luminance;

a second plurality of pixels particularly sensitive to chrominance, wherein said first plurality of pixels comprises more than two-thirds of a total number of pixels of said image sensor, wherein at least some pixels included in said first plurality pixels are adjacent to at least one of said pixels included in said second plurality of pixels, wherein said second plurality of pixels are arranged in groups of four, and wherein each of said groups comprises first and second pixels having sensitivity to a first spectral band, a third pixel having sensitivity to a second spectral band, and a fourth pixel having sensitivity to a third spectral band.

37. The device of claim 36, further comprising a plurality of filters, wherein a first filter type allows light within said first spectral band to reach said first and second pixels, wherein a second filter type allows light within said second spectral band to reach said third pixels, and wherein a third filter type allows light within said third spectral band to reach said fourth pixels.

38. The device of claim 36, wherein said second plurality of pixels comprises less than 10% of said total number of pixels of said image sensor.

39. A device for color imaging in low light level conditions, comprising:

an image sensor, wherein said image sensor comprises;

a first plurality of pixels particularly sensitive to luminance;

a second plurality of pixels particularly sensitive to chrominance, wherein said first plurality of pixels comprises more than two-thirds of a total number of pixels of said image sensor, wherein at least some pixels included in said first plurality pixels are adjacent to at least one of said pixels included in said second plurality of pixels, and wherein said second plurality of pixels are pseudo-randomly distributed about said image sensor.

40. The device of claim 39, wherein said second plurality of pixels comprises less than 10% of said total number of pixels of said image sensor.

41. A device for color imaging in low light level conditions, comprising:

an image sensor, wherein said image sensor comprises;

a first plurality of pixels particularly sensitive to luminance;

a second plurality of pixels particularly sensitive to chrominance, wherein said first plurality of pixels comprises more than two-thirds of a total number of pixels of said image sensor, and wherein at least some pixels included in said first plurality pixels are adjacent to at least one of said pixels included in said second plurality of pixels; and

a switch, wherein said switch is operable to selectively turn off a signal from said second plurality of pixels.

42. The device of claim 41, wherein said second plurality of pixels comprises less than 10% of said total number of pixels of said image sensor.

43. A device for color imaging in low light level conditions, comprising:

an image sensor, wherein said image sensor comprises;

a first plurality of pixels particularly sensitive to luminance;

a second plurality of pixels particularly sensitive to chrominance, wherein said first plurality of pixels comprises more than half of a total number of pixels of said image sensor, wherein said second plurality of pixels comprises less than 4% of said total number of pixels of said image sensor.

44. A low light level image sensor, comprising:

a first image sensor comprising an array of pixels;

a color filter, wherein a first plurality of said pixels are provided with image information filtered to within a first spectrum defined by a first band pass filter, wherein a second plurality of said pixels are provided with image information filtered to within a second spectrum defined by a second band pass filter, wherein a third plurality of said pixels are provided with image information filtered to within a third spectrum by a third band pass filter, and wherein said first, second, and third pluralities of pixels provided with image information filtered to within said first, second or third spectrum comprise less than one-third of a total number of said pixels of said image sensor, and wherein said first, second and third pluralities of pixels provide at least color information.

45. The low light level image sensor of claim 44, wherein a majority of said pixels of said image sensor receive image information at wavelengths greater than 780 nanometers.

46. The low light level image sensor of claim 44, wherein said first, second, and third pluralities of pixels provided with filtered image information are distributed randomly among said pixels of said image sensor.

47. The low light image sensor of claim 44, wherein said first, second and third pluralities of pixels provided with image information filtered to within said first, second or third spectrum are distributed about said image sensor in groups comprising at least one pixel of each of said first, second and third pluralities of pixels, and wherein at least most of said groups are surrounded by a plurality of pixels that are not provided with image information filtered to within said first, second or third spectrum.

48. The low light image sensor of claim 44, further comprising a switch, wherein signals from said first, second, and third pluralities of pixels can be selectively switched off.

49. The low light image sensor of claim 44, wherein said first, second, and third pluralities of pixels are interspersed among pixels of said image sensor that are provided with image information that is not filtered to within one of said first, second and third spectrums such that a majority of pixels included in said first, second, and third pluralities of pixels has at least one pixel that is provided with image information that is not filtered to within one of said first, second and third spectrums as a neighbor pixel.

50. The device of claim 44, wherein at least some of said pixels included in said second plurality of pixels are arranged in groups, wherein at least some of said groups are surrounded by pixels not included in said first, second or third pluralities of pixels.

51. The low light level image sensor of claim 44, wherein said first, second, and third pluralities of pixels provided with image information filtered to within said first, second, or third spectrum comprise less than 10% of a total number of said pixels of said image sensor.

52. A low light level image sensor, comprising:

a first image sensor comprising an array of pixels;

a color filter, wherein a first plurality of said pixels are provided with image information filtered by a first band pass filter, wherein a second plurality of said pixels are provided with image information filtered by a second band pass filter, wherein a third plurality of pixels are provided with image information filtered by a third band pass filter, wherein said pixels are arranged in groups of pixels including at least one pixel from each of said first, second and third pluralities of pixels, and wherein a plurality of said groups are surrounded by a plurality of pixels that do not receive image information filtered by a first, second or third band pass filter.

53. The low light image sensor of claim 52, wherein said first, second, and third pluralities of pixels comprise less than 10% of a total number of pixels included in said first image sensor.