RGBW sensor array

• US 8,878,967 B2
• Filed: 10/11/2010
• Issued: 11/04/2014
• Est. Priority Date: 03/05/2007
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Susanu, George, Petrescu, Stefan, Nanu, Florin, Capata, Adrian, Corcoran, Peter
• Primary Examiner(s): HANNETT, JAMES M
• Application Number:
  US12/901,577
• Publication Number:
  US 20110102638A1
• Time in Patent Office: 1,485 Days
• Field of Search: 348/272
• US Class Current: 348/272
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10144 Varying exposure
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/133 including elements passing panchromatic light, e.g. filters passing white light
• CPC Class Codes: H04N 25/589 with different integration times, e.g. short and long exposures

Abstract

A color filter enhancement method for a portable digital image acquisition device includes digitally exposing color pixels of a color sensor array for a first digital exposure duration and digitally exposing white pixels of a color sensor array for a second digital exposure time shorter than the first digital exposure duration. A color filter enhanced digital image is generated using data from both the color pixels exposed for the first digital exposure duration and the white pixels exposed for the second digital exposure duration.

Claims

1. A color filter enhancement method for a portable digital image acquisition device, comprising:

using optics, a color sensor array and a processor to acquire and process digital images;

obtaining a first relatively underexposed and sharp image of a scene (“

sharp image”

) by exposing a first set of pixels of the color sensor array;

obtaining a second relatively well exposed and blurred image of the same scene (“

blurred image”

) by exposing a second set of pixels of the color sensor array for a longer duration than the sharp image; and

applying color information from the blurred image to the sharp image, thereby constructing a color-enhanced version of the sharp image;

wherein applying the color information from the blurred image to the sharp image includes changing one or more pixels of the sharp image based on the color information of the blurred image.

2. The method of claim 1, wherein obtaining the blurred image comprises digitally exposing color pixels of the color sensor array for a first digital exposure duration;

and wherein obtaining the sharp image comprises digitally exposing white pixels of the color sensor array for a second digital exposure duration shorter than the first digital exposure duration.

3. The method of claim 2, wherein the constructing of the color-enhanced version of the sharp image comprises using data from both the color pixels exposed for the first digital exposure duration and the white pixels exposed for the second digital exposure duration.

4. The method of claim 2, wherein the digitally exposing of the color pixels and the white pixels for different exposure times comprises clocking the color pixels and the white pixels independently.

5. The method of claim 2, wherein the digitally-exposing of the color pixels and the white pixels for different exposure times comprises including sensor data over different temporal ranges.

6. The method of claim 5, wherein the different temporal ranges are overlapping.

7. The method of claim 6, wherein a first temporal range corresponding to the digitally-exposing of the color pixels comprises an entire second temporal range corresponding to the digitally-exposing of the white pixels.

8. The method of claim 2, wherein the color pixels comprise greater motion blurring effect than the white pixels due to the color pixels being digitally-exposed for a longer duration than the white pixels, and the method further comprises compensating blurring in the color pixels using less-blurred data from the white pixels.

9. The method of claim 2, wherein at least a segment of the first digital exposure duration and the second digital exposure duration temporally overlap.

10. The method of claim 2, wherein the second digital exposure duration comprises not more than half of the first digital exposure duration.

11. The method of claim 2, wherein the second digital exposure duration comprises approximately a third of the first digital exposure duration.

12. The method of claim 1, wherein the color sensor array comprises a CMOS-based sensor.

13. The method of claim 1, further comprising storing, transmitting, or displaying the color-enhanced version of the sharp image, or a further processed version, or combinations thereof.

40. The method of claim 1, further comprising:

applying luminance information from the blurred image to the sharp image, thereby constructing a luminance-enhanced version of the sharp image;

wherein applying the luminance information from the blurred image to the sharp image includes changing one or more pixels of the sharp image based on the luminance information of the blurred image.

41. The method of claim 40, wherein:

obtaining the blurred image comprises digitally exposing color pixels of the color sensor array for a first digital exposure duration;

obtaining the sharp image comprises digitally exposing white pixels of the color sensor array for a second digital exposure duration shorter than the first digital exposure duration; and

constructing the luminance-enhanced version of the sharp image comprises using data from both the color pixels exposed for the first digital exposure duration and the white pixels exposed for the second digital exposure duration.

42. The method of claim 40, further comprising storing, transmitting, or displaying the luminance-enhanced version of the sharp image, or a further processed version, or combinations thereof.

14. A portable digital image acquisition device, comprising:

optics and a color sensor array for acquiring a color digital image;

a processor;

one or more processor-readable media having code embedded therein for programming the processor to perform a color filter enhancement method that comprises the following;

obtaining a first relatively underexposed and sharp image of a scene (“

sharp image”

) by exposing a first set of pixels of the sensor array;

obtaining a second relatively well exposed and blurred image of the same scene (“

blurred image”

) by exposing a second set of pixels of the color sensor array for a longer duration than the sharp image; and

applying color information from the blurred image to the sharp image, thereby constructing a color-enhanced of the sharp image;

wherein applying the color information from the blurred image to the sharp image includes changing one or more pixels of the sharp image based on the color information of the blurred image.

15. The device of claim 14, wherein obtaining the blurred image comprises digitally exposing color pixels of the color sensor array for a first digital exposure duration;

and wherein obtaining the sharp image comprises digitally exposing white pixels of the color sensor array for a second digital exposure duration shorter than the first digital exposure duration.

16. The device of claim 15, wherein the constructing of the color-enhanced version of the sharp image comprises using data from both the color pixels exposed for the first digital exposure duration and the white pixels exposed for the second digital exposure duration.

17. The device of claim 15, wherein the digitally exposing of the color pixels and the white pixels for different exposure times comprises clocking the color pixels and the white pixels independently.

18. The device of claim 15, wherein the digitally-exposing of the color pixels and the white pixels for different exposure times comprises including sensor data over different temporal ranges.

19. The device of claim 18, wherein the different temporal ranges are overlapping.

20. The device of claim 19, wherein a first temporal range corresponding to the digitally-exposing of the color pixels comprises an entire second temporal range corresponding to the digitally-exposing of the white pixels.

21. The device of claim 15, wherein the color pixels comprise greater motion blurring effect than the white pixels due to the color pixels being digitally-exposed for a longer duration than the white pixels, and the method further comprises compensating blurring in the color pixels using less-blurred data from the white pixels.

22. The device of claim 15, wherein at least a segment of the first digital exposure duration and the second digital exposure duration temporally overlap.

23. The device of claim 15, wherein the second digital exposure duration comprises not more than half of the first digital exposure duration.

24. The device of claim 15, wherein the second digital exposure duration comprises approximately a third of the first digital exposure duration.

25. The device of claim 14, wherein the color sensor array comprises a CMOS-based sensor.

26. The device of claim 14, wherein the method further comprises storing, transmitting, or displaying the color-enhanced version of the sharp image, or a further processed version, or combinations thereof.

43. The device of claim 14, wherein the method further comprises:

applying luminance information from the blurred image to the sharp image, thereby constructing a luminance-enhanced version of the sharp image;

wherein applying the luminance information from the blurred image to the sharp image includes changing one or more pixels of the sharp image based on the luminance information of the blurred image.

44. The device of claim 43, wherein:

obtaining the blurred image comprises digitally exposing color pixels of the color sensor array for a first digital exposure duration;

obtaining the sharp image comprises digitally exposing white pixels of the color sensor array for a second digital exposure duration shorter than the first digital exposure duration; and

constructing of the luminance-enhanced version of the sharp image comprises using data from both the color pixels exposed for the first digital exposure duration and the white pixels exposed for the second digital exposure duration.

45. The device of claim 43, wherein the method further comprises storing, transmitting, or displaying the luminance-enhanced version of the sharp image, or a further processed version, or combinations thereof.

27. One or more non-transitory processor-readable media having code embedded therein for programming the processor to perform a color filter enhancement method that comprises:

obtaining a first relatively underexposed and sharp image of a scene (“

sharp image”

) by exposing a first set of pixels of the color sensor array;

obtaining a second relatively well exposed and blurred image of the same scene (“

blurred image”

) by exposing a second set of pixels of the color sensor array for a longer duration than the sharp image; and

applying color information from the blurred image to the sharp image, thereby constructing a color-enhanced version of the sharp image;

wherein applying the color information from the blurred image to the sharp image includes changing one or more pixels of the sharp image based on the color information of the blurred image.

28. The one or more non-transitory processor-readable media of claim 27, wherein obtaining the blurred image comprises digitally exposing color pixels of the color sensor array for a first digital exposure duration;

and wherein obtaining the sharp image comprises digitally exposing white pixels of the color sensor array for a second digital exposure duration shorter than the first digital exposure duration.

29. The one or more non-transitory processor-readable media of claim 28, wherein the constructing of the color-enhanced version of the sharp image comprises using data from both the color pixels exposed for the first digital exposure duration and the white pixels exposed for the second digital exposure duration.

30. The one or more non-transitory processor-readable media of claim 28, wherein the digitally exposing of the color pixels and the white pixels for different exposure times comprises clocking the color pixels and the white pixels independently.

31. The one or more non-transitory processor-readable media of claim 28, wherein the digitally-exposing of the color pixels and the white pixels for different exposure times comprises including sensor data over different temporal ranges.

32. The one or more non-transitory processor-readable media of claim 31, wherein the different temporal ranges are overlapping.

33. The one or more non-transitory processor-readable media of claim 32, wherein a first temporal range corresponding to the digitally-exposing of the color pixels comprises an entire second temporal range corresponding to the digitally-exposing of the white pixels.

34. The one or more non-transitory processor-readable media of claim 28, wherein the color pixels comprise greater motion blurring effect than the white pixels due to the color pixels being digitally-exposed for a longer duration than the white pixels, and the method further comprises compensating blurring in the color pixels using less-blurred data from the white pixels.

35. The one or more non-transitory processor-readable media of claim 28, wherein at least a segment of the first digital exposure duration and the second digital exposure duration temporally overlap.

36. The one or more non-transitory processor-readable media of claim 28, wherein the second digital exposure duration comprises not more than half of the first digital exposure duration.

37. The one or more non-transitory processor-readable media of claim 28, wherein the second digital exposure duration comprises approximately a third of the first digital exposure duration.

38. The one or more non-transitory processor-readable media of claim 28, wherein the color sensor array comprises a CMOS-based sensor.

39. The one or more non-transitory processor-readable media of claim 28, wherein the method further comprises storing, transmitting, or displaying the color-enhanced version of the sharp image, or a further processed version, or combinations thereof.

46. The one or more non-transitory processor-readable media of claim 28, wherein the method further comprises:

applying luminance information from the blurred image to the sharp image, thereby constructing a luminance-enhanced version of the sharp image;

wherein applying the luminance information from the blurred image to the sharp image includes changing one or more pixels of the sharp image based on the luminance information of the blurred image.

47. The one or more non-transitory processor-readable media of claim 46, wherein:

obtaining the blurred image comprises digitally exposing color pixels of the color sensor array for a first digital exposure duration;

obtaining the sharp image comprises digitally exposing white pixels of the color sensor array for a second digital exposure duration shorter than the first digital exposure duration; and

constructing of the luminance-enhanced version of the sharp image comprises using data from both the color pixels exposed for the first digital exposure duration and the white pixels exposed for the second digital exposure duration.

48. The one or more non-transitory processor-readable media of claim 46, wherein the method further comprises storing, transmitting, or displaying the luminance-enhanced version of the sharp image, or a further processed version, or combinations thereof.