DIGITAL IMAGE PROCESSING AND SYSTEMS INCORPORATING THE SAME

• US 20100195908A1
• Filed: 02/01/2010
• Published: 08/05/2010
• Est. Priority Date: 02/02/2009
• Status: Active Grant

• Current Assignee: HL Klemove Corporation
• Original Assignee: Gentex Corporation
• Inventors: Andrus, Jeremy C., Bechtel, Jon H., Sherman, Tom B.
• Granted Patent: US 9,866,805 B2
• Time in Patent Office: Days
• US Class Current: 382/167
• CPC Class Codes: H04N 21/4318 by altering the content in the rendering process, e.g. blanking, blurring or masking an image region image enhancement or restoration in general G06T5/00
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/84 for processing colour signals
• CPC Class Codes: H04N 5/213 Circuitry for suppressing or minimising impulsive noise for suppressing or minimising disturbance in image signal generation H04N23/81
• CPC Class Codes: H04N 5/57 Control of contrast or brightness

Abstract

In at least one embodiment, a spectrally mozaiced digital image is provided with missing color data imputed for each pixel. At least one embodiment of the present invention is related to circuitry configured to perform at least a portion of the calculations related to demosaicing a high dynamic range image.

Claims

1. An apparatus, comprising:

an algorithm for calculating missing color data, said algorithm is configured to receive data associated with a spectrally mosaiced, high dynamic range, digital image and to calculate missing color data, said algorithm generates a plurality of patterns with all possible ones between one (1) and (n−

1) for a pattern image comprising n pixels to detect features of said spectrally mosaiced, high dynamic range, digital image.

2. An apparatus as in claim 1 wherein said detected features are at least in part used to determine selection of an interpolation equation to calculate missing color data for one or more pixels within said pattern image.

3. An apparatus as in claim 1 wherein said algorithm is further configured to generate a set of binary values indicative of the relative brightness of a reference pixel compared to the brightnesses of other pixels in said pattern array.

4. An apparatus as in claim 3 wherein said algorithm is further configured to analyze said set of binary data to detect patterns in said data.

5. An apparatus as in claim 4 wherein said algorithm is further configured to select an interpolation equation based at least in part on detected patterns in said data.

6. An apparatus as in claim 3 wherein said algorithm is further configured such that more than one pixel is used as a reference pixel to generate more than one set of binary data.

7. An apparatus as in claim 1, wherein said spectrally mosaiced high dynamic range digital image comprises a plurality of red spectrally filtered pixels, a plurality of green spectrally filtered pixels and a plurality of blue spectrally filtered pixels.

8. An apparatus as in claim 7 wherein said missing color data is associated with a red spectrally filtered pixel, green and blue color data are calculated.

9. An apparatus as in claim 7 wherein said missing color data is associated with a green spectrally filtered pixel, red and blue color data are calculated.

10. An apparatus as in claim 7 wherein said missing color data is associated with a blue spectrally filtered pixel, red and green color data are calculated.

11. An apparatus as in claim 10 configured to receive data associated with a first series of spectrally mosaiced, high dynamic range, digital images from an imaging device and output a second series of digital images having missing color data imputed.

12. An apparatus as in claim 11 wherein data associated with said first series of spectrally mosaiced, high dynamic range, digital images is received at a rate greater than, or equal to, twenty-five images per second.

13. An apparatus as in claim 12 wherein data associated with a second series of spectrally demosaiced, high dynamic range, digital images is output at a rate greater than, or equal to, twenty-five frames per second.

14. An apparatus, comprising:

electronic circuitry for performing at least a portion of the calculations associated with an algorithm for calculating missing color data, said algorithm is configured to receive data associated with a spectrally mosaiced, high dynamic range, digital image and to calculate missing color data, said algorithm implements a pattern based logic comparison wherein a pixel value selected as a reference pixel is compared against other pixel values in a pattern array to generate a set of binary values that indicate how the brightness of said reference pixel compares with the brightnesses of other pixels in the pattern array.

15. An apparatus as in claim 14 wherein said detected features are at least in part used to determine selection of an equation to interpolation calculate missing color data for one or more pixels within said pattern image.

16. An apparatus as in claim 14 wherein said algorithm is further configured to analyze said set of binary data to detect patterns in said data.

17. An apparatus as in claim 16 wherein said algorithm is further configured to select an interpolation equation based at least in part on detected patterns in said data.

18. An apparatus as in claim 14 wherein said algorithm is further configured such that more than one pixel is used as a reference pixel to generate more than one set of binary data.

19. An apparatus as in claim 14, wherein said spectrally mosaiced high dynamic range digital image comprises a plurality of red spectrally filtered pixels, a plurality of green spectrally filtered pixels and a plurality of blue spectrally filtered pixels.

20. An apparatus as in claim 19 wherein said missing color data is associated with a red spectrally filtered pixel, green and blue color data are calculated.

21. An apparatus as in claim 19 wherein said missing color data is associated with a green spectrally filtered pixel, red and blue color data are calculated.

22. An apparatus as in claim 19 wherein said missing color data is associated with a blue spectrally filtered pixel, red and green color data are calculated.

23. An apparatus as in claim 22 configured to receive data associated with a first series of spectrally mosaiced, high dynamic range, digital images from an imaging device and output a second series of digital images having missing color data imputed.

24. An apparatus as in claim 23 wherein data associated with said first series of spectrally mosaiced, high dynamic range, digital images is received at a rate greater than, or equal to, twenty-five images per second.

25. An apparatus as in claim 24 wherein data associated with a second series of spectrally demosaiced, high dynamic range, digital images is output at a rate greater than, or equal to, twenty-five frames per second.

26. An apparatus, comprising:

an algorithm for calculating missing color data, said algorithm is configured to receive data associated with a spectrally mosaiced, high dynamic range, digital image and to calculate missing color data, said algorithm generates a pattern of results based on comparisons of pixels in a pattern array with a given pixel selected as a reference pixel in the pattern array, said pattern of results is analyzed to detect features of said spectrally mosaiced, high dynamic range, digital image.

27. An apparatus as in claim 26 wherein said detected features are at least in part used to determine selection of an interpolation equation to calculate missing color data for one or more pixels within said pattern image.

28. An apparatus as in claim 26 wherein said algorithm is further configured to generate a set of binary values indicative of the relative brightness of a reference pixel compared to the brightnesses of other pixels in said pattern array.

29. An apparatus as in claim 28 wherein said algorithm is further configured to analyze said set of binary data to detect patterns in said data.

30. An apparatus as in claim 29 wherein said algorithm is further configured to select an interpolation equation based at least in part on detected patterns in said data.

31. An apparatus as in claim 28 wherein said algorithm is further configured such that more than one pixel is used as a reference pixel to generate more than one set of binary data.

32. An apparatus as in claim 26, wherein said spectrally mosaiced high dynamic range digital image comprises a plurality of red spectrally filtered pixels, a plurality of green spectrally filtered pixels and a plurality of blue spectrally filtered pixels.

33. An apparatus as in claim 32 wherein said missing color data is associated with a red spectrally filtered pixel, green and blue color data are calculated.

34. An apparatus as in claim 32 wherein said missing color data is associated with a green spectrally filtered pixel, red and blue color data are calculated.

35. An apparatus as in claim 32 wherein said missing color data is associated with a blue spectrally filtered pixel, red and green color data are calculated.

36. An apparatus as in claim 35 configured to receive data associated with a first series of spectrally mosaiced, high dynamic range, digital images from an imaging device and output a second series of digital images having missing color data imputed.

37. An apparatus as in claim 36 wherein data associated with said first series of spectrally mosaiced, high dynamic range, digital images is received at a rate greater than, or equal to, twenty-five images per second.

38. An apparatus as in claim 37 wherein data associated with a second series of spectrally demosaiced, high dynamic range, digital images is output at a rate greater than, or equal to, twenty-five frames per second.

39. An apparatus, comprising:

an algorithm for calculating missing color data, said algorithm is configured to receive data associated with a spectrally mosaiced, high dynamic range, digital image and to calculate missing color data, said algorithm generates two or more patterns of results based on comparisons of pixels in a pattern array with different pixels selected as a reference pixel in the pattern array, said patterns of results are analyzed to detect features of said spectrally mosaiced, high dynamic range, digital image.

40. An apparatus as in claim 39 wherein said detected features are at least in part used to determine selection of an interpolation equation to calculate missing color data for one or more pixels within said pattern image.

41. An apparatus as in claim 39 wherein said algorithm is further configured to generate a set of binary values indicative of the relative brightness of a reference pixel compared to the brightnesses of other pixels in said pattern array.

42. An apparatus as in claim 41 wherein said algorithm is further configured to analyze said set of binary data to detect patterns in said data.

43. An apparatus as in claim 42 wherein said algorithm is further configured to select an interpolation equation based at least in part on detected patterns in said data.

44. An apparatus as in claim 41 wherein said algorithm is further configured such that more than one pixel is used as a reference pixel to generate more than one set of binary data.

45. An apparatus as in claim 39, wherein said spectrally mosaiced high dynamic range digital image comprises a plurality of red spectrally filtered pixels, a plurality of green spectrally filtered pixels and a plurality of blue spectrally filtered pixels.

46. An apparatus as in claim 45 wherein said missing color data is associated with a red spectrally filtered pixel, green and blue color data are calculated.

47. An apparatus as in claim 45 wherein said missing color data is associated with a green spectrally filtered pixel, red and blue color data are calculated.

48. An apparatus as in claim 45 wherein said missing color data is associated with a blue spectrally filtered pixel, red and green color data are calculated.

49. An apparatus as in claim 48 configured to receive data associated with a first series of spectrally mosaiced, high dynamic range, digital images from an imaging device and output a second series of digital images having missing color data imputed.

50. An apparatus as in claim 49 wherein data associated with said first series of spectrally mosaiced, high dynamic range, digital images is received at a rate greater than, or equal to, twenty-five images per second.

51. An apparatus as in claim 50 wherein data associated with a second series of spectrally demosaiced, high dynamic range, digital images is output at a rate greater than, or equal to, twenty-five frames per second.

52. An apparatus, comprising:

an algorithm for calculating missing color data, said algorithm is configured to receive data associated with a spectrally mosaiced, high dynamic range, digital image and to calculate missing color data, said algorithm generates patterns of results based on comparisons of pixels in a pattern array with a different pixel from the pattern array selected as a reference pixel in the pattern array so that every pixel in the array is used as a reference pixel, said patterns of results are analyzed to detect features of said spectrally mosaiced, high dynamic range, digital image.

53. An apparatus as in claim 52 wherein said detected features are at least in part used to determine selection of an interpolation equation to calculate missing color data for one or more pixels within said pattern image.

54. An apparatus as in claim 52 wherein said algorithm is further configured to generate a set of binary values indicative of the relative brightness of a reference pixel compared to the brightnesses of other pixels in said pattern array.

55. An apparatus as in claim 54 wherein said algorithm is further configured to analyze said set of binary data to detect patterns in said data.

56. An apparatus as in claim 55 wherein said algorithm is further configured to select an interpolation equation based at least in part on detected patterns in said data.

57. An apparatus as in claim 54 wherein said algorithm is further configured such that more than one pixel is used as a reference pixel to generate more than one set of binary data.

58. An apparatus as in claim 52, wherein said spectrally mosaiced high dynamic range digital image comprises a plurality of red spectrally filtered pixels, a plurality of green spectrally filtered pixels and a plurality of blue spectrally filtered pixels.

59. An apparatus as in claim 58 wherein said missing color data is associated with a red spectrally filtered pixel, green and blue color data are calculated.

60. An apparatus as in claim 58 wherein said missing color data is associated with a green spectrally filtered pixel, red and blue color data are calculated.

61. An apparatus as in claim 58 wherein said missing color data is associated with a blue spectrally filtered pixel, red and green color data are calculated.

62. An apparatus as in claim 61 configured to receive data associated with a first series of spectrally mosaiced, high dynamic range, digital images from an imaging device and output a second series of digital images having missing color data imputed.

63. An apparatus as in claim 62 wherein data associated with said first series of spectrally mosaiced, high dynamic range, digital images is received at a rate greater than, or equal to, twenty-five images per second.

64. An apparatus as in claim 63 wherein data associated with a second series of spectrally demosaiced, high dynamic range, digital images is output at a rate greater than, or equal to, twenty-five frames per second.

65. An apparatus, comprising:

an algorithm for calculating missing color data, said algorithm is configured to receive a spectrally mosaiced, high dynamic range, digital image and to calculate missing color data, said algorithm implements a pattern based logic comparison wherein a pixel value selected as a reference pixel is compared against other pixel values in a pattern array to indicate how the brightness of said reference pixel compares with the brightnesses of other pixels in the pattern array.

66. An apparatus as in claim 65 wherein said detected features are at least in part used to determine selection of an interpolation equation to calculate missing color data for one or more pixels within said pattern image.

67. An apparatus as in claim 65 wherein said algorithm is further configured to analyze said set of binary data to detect patterns in said data.

68. An apparatus as in claim 67 wherein said algorithm is further configured to select an interpolation equation based at least in part on detected patterns in said data.

69. An apparatus as in claim 65 wherein said algorithm is further configured such that more than one pixel is used as a reference pixel to generate more than one set of binary data.

70. An apparatus as in claim 65, wherein said spectrally mosaiced high dynamic range digital image comprises a plurality of red spectrally filtered pixels, a plurality of green spectrally filtered pixels and a plurality of blue spectrally filtered pixels.

71. An apparatus as in claim 70 wherein said missing color data is associated with a red spectrally filtered pixel, green and blue color data are calculated.

72. An apparatus as in claim 70 wherein said missing color data is associated with a green spectrally filtered pixel, red and blue color data are calculated.

73. An apparatus as in claim 70 wherein said missing color data is associated with a blue spectrally filtered pixel, red and green color data are calculated.

74. An apparatus as in claim 73 configured to receive data associated with a first series of spectrally mosaiced, high dynamic range, digital images from an imaging device and output a second series of digital images having missing color data imputed.

75. An apparatus as in claim 74 wherein data associated with said first series of spectrally mosaiced, high dynamic range, digital images is received at a rate greater than, or equal to, twenty-five images per second.

76. An apparatus as in claim 75 wherein data associated with a second series of spectrally demosaiced, high dynamic range, digital images is output at a rate greater than, or equal to, twenty-five frames per second.

77. An apparatus, comprising:

electronic circuitry for performing at least a portion of the calculations associated with an algorithm for calculating missing color data, said algorithm is configured to receive data associated with a spectrally mosaiced digital image and to calculate missing color data, said algorithm implements an average based comparison wherein a pixel value selected as a reference pixel is compared against the average of pixel values in a pattern array to detect features.

78. An apparatus as in claim 77 wherein said detected features are at least in part used to determine selection of an interpolation equation to calculate missing color data for one or more pixels within said pattern image.

79. An apparatus as in claim 77 wherein said algorithm is further configured such that more than one pixel is used as a reference pixel.

80. An apparatus as in claim 77, wherein said spectrally mosaiced digital image comprises a plurality of red spectrally filtered pixels, a plurality of green spectrally filtered pixels and a plurality of blue spectrally filtered pixels.

81. An apparatus as in claim 80 wherein said missing color data is associated with a red spectrally filtered pixel, green and blue color data are calculated.

82. An apparatus as in claim 80 wherein said missing color data is associated with a green spectrally filtered pixel, red and blue color data are calculated.

83. An apparatus as in claim 80 wherein said missing color data is associated with a blue spectrally filtered pixel, red and green color data are calculated.

84. An apparatus as in claim 83 configured to receive data associated with a first series of spectrally mosaiced, high dynamic range, digital images from an imaging device and output a second series of digital images having missing color data imputed.

85. An apparatus as in claim 84 wherein data associated with said first series of spectrally mosaiced, high dynamic range, digital images is received at a rate greater than, or equal to, twenty-five images per second.

86. An apparatus as in claim 85 wherein data associated with a second series of spectrally demosaiced, high dynamic range, digital images is output at a rate greater than, or equal to, twenty-five frames per second.