Optimized image processing for wavefront coded imaging systems

• US 20030169944A1
• Filed: 02/27/2003
• Published: 09/11/2003
• Est. Priority Date: 02/27/2002
• Status: Active Grant

• Current Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Original Assignee: OmniVision CDM Optics, Inc.
• Inventors: Dowski, Edward Raymond Jr., Wach, Hans B., Macon, Ashley K., Johnson, Gregory Edward
• Granted Patent: US 7,379,613 B2
• Time in Patent Office: Days
• US Class Current: 382/279
• CPC Class Codes: G02B 27/0025 for optical correction, e.g. distorsion, aberration
• CPC Class Codes: G02B 27/46 Systems using spatial filters
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

An image processing method includes the steps of wavefront coding a wavefront that forms an optical image, converting the optical image to a data stream, and processing the data stream with a filter kernel to reverse effects of wavefront coding and generate a final image. By example, the filter set kernel may be a reduced filter set kernel, or a color-specific kernel. Methods and systems are also disclosed for processing color images, such as by separating color and spatial information into separate channels. Methods and systems herein are for example useful in forming electronic devices with reduced opto-mechanical, opto-electronic and processing complexity or cost.

Claims

1. An image processing method, comprising the steps of:

wavefront coding a wavefront that forms an optical image;

converting the optical image to a data stream;

processing the data stream with a reduced set filter kernel to reverse effects of wavefront coding and generate a final image.

2. The image processing method of claim 1, the step of processing comprising the step of utilizing a filter kernel that is complimentary to an MTF of the optical image.

3. The image processing method of claim 2, the steps of wavefront coding, converting and processing occurring such that the MTF is spatially correlated to mathematical processing of the data stream with the reduced set filter kernel.

4. The method of claim 1, the step of wavefront coding comprising the step of utilizing an aspheric wavefront coded optical element.

5. The method of claim 1, the step of wavefront coding comprising wavefront coding the wavefront with constant path profile optics.

6. The method of claim 5, further comprising the step of formulating the reduced set filter kernel to an MTF of the optical image resulting from phase modification of the wavefront through the constant path profile optics.

7. The method of claim 1, the step of processing comprising the step of processing the image, for each pixel, with filter tap logic consisting of a shifter.

8. The method of claim 1, the step of processing comprising the step of processing the image, for each pixel, with filer tap logic consisting of a shifter and an adder.

9. The method of claim 1, the step of processing comprising the step of processing the data with a reduced set filter kernel consisting of one or more of (a) power of two coefficients, (b) sums of power of two coefficients, and (c) differences of power of two coefficients.

10. The method of claim 1, the step of processing comprising the step of processing the data with a reduced set filter kernel consisting of a plurality or regions wherein at least one of the regions has zero values.

11. The method of claim 10, the step of wavefront coding comprising the step of wavefront coding the wavefront such that a PSF of the optical image spatially correlates to the regions of the reduced set filter kernel, wherein a substantial amount of information of the PSF is within the final image.

12. The method of claim 1, the step of converting the optical image to a data stream comprising outputting data from a detector in a non-rectilinear format.

13. The method of claim 12, the format comprising one of a diagonal readout and a circular readout.

14. The method of claim 12, the step of processing comprising processing the data stream with a reduced set filter kernel matched to the non-recitilinear format.

15. The method of claim 14, further comprising the step of remapping data to a rectilinear format.

16. The method of claim 12, the step of wavefront coding comprising wavefront coding the wavefront such that the optical image at the detector substantially coincides with the non-rectilinear format of the detector.

17. The method of claim 1, the step of processing comprising the steps of utilizing a reduced set gradient filter kernel and a series of single-shift differential taps.

18. The method of claim 17, the step of processing comprising the step of processing a single summation for any coefficient value of the filter kernel.

19. The method of claim 17, the step of wavefront coding comprising the step of wavefront coding the wavefront such that a PSF of the optical image substantially contains information of the optical image within a spatial pattern corresponding to the filter kernel.

20. The method of claim 1, the step of processing comprising the steps of utilizing a reduced set scaling filter kernel and a series of scaling taps.

21. The method of claim 20, the step of processing comprising the step of processing with a single summation for any coefficient value of the filter kernel.

22. The method of claim 20, the step of wavefront coding comprising the step of wavefront coding the wavefront such that a PSF of the optical image substantially contains information within a spatial pattern corresponding to the filter kernel.

23. The method of claim 1, the step of processing comprising the steps of utilizing a reduced set distributive filter kernel and a series of distributive property scale-add taps.

24. The method of claim 23, the step of processing comprising the step of processing a single summation for any coefficient value of the filter kernel.

25. The method of claim 23, the step of wavefront coding comprising the step of wavefront coding the wavefront such that a PSF of the optical image substantially contains information of the optical image within a spatial pattern corresponding to the filter kernel.

26. The method of claim 1, the step of processing comprising the steps of utilizing a reduced set distributive filter kernel and a series of distributive property add-scale taps.

27. The method of claim 26, the step of processing comprising the step of processing with a single summation for any coefficient value of the filter kernel.

28. The method of claim 26, the step of wavefront coding comprising the step of wavefront coding the wavefront such that a PSF of the optical image substantially contains information of the optical image within a spatial pattern corresponding to the filter kernel.

29. The method of claim 1, further comprising the step of selecting a desired frequency response of the final image by selecting the reduced set filter kernel.

30. The method of claim 1, further comprising the step of modifying one of the wavefront coding, converting and processing steps and then optimizing and repeating one other of the wavefront coding, converting and processing steps.

31. The method of claim 30, the step of processing comprising utilizing a reduced set filter kernel comprising a weighted matrix.

32. An image processing method, comprising the steps of:

wavefront coding a wavefront that forms an optical image;

converting the optical image to a data stream;

processing the data stream with a color-specific filter kernel to reverse effects of wavefront coding and generate a final image.

33. The method of claim 32, the step of converting the optical image to a data stream comprising outputting color data from a color digital detector into separate channels.

34. An image processing method, comprising the steps of:

wavefront coding a wavefront that forms an optical image;

converting the optical image to a data stream;

colorspace converting the datastream;

separating spatial information and color information of the colorspace converted datastream into one or more separate channels;

deblurring one or both of the spatial information and the color information;

recombining the channels to recombine deblurred spatial information with deblurred color information; and

colorspace converting the recombined deblurred spatial and color information to generate an output image.

35. The method of claim 34, further comprising a first step of filtering noise from the data stream.

36. The method of claim 35, wherein the step of wavefront coding comprises generating a MTF of the optical image such that spatial correlation exists with the first step of filtering noise.

37. The method of claim 34, further comprising a second step of filtering noise from the recombined deblurred spatial and color information.

38. The method of claim 37, the second step of filtering noise comprises utilizing a complement of an MTF of the optical image.

39. The method of claim 34, the step of colorspace converting the datastream comprising the step of spatially varying image data with a dynamic color space process.

40. An optical imaging system for generating an optical image, comprising:

a wavefront coding element that codes a wavefront forming the optical image;

a detector for converting the optical image to a data stream; and

an image processor for processing the data stream with a reduced set filter kernel to reverse effects of wavefront coding and generate a final image.

41. The optical imaging system of claim 40, the filter kernel being spatially complimentary to a PSF of the optical image.

42. The optical imaging system of claim 40, a spatial frequency domain version of the filter kernel being complimentary to a MTF of the optical image.

43. The optical imaging system of claim 42, the wavefront coding element, detector and image processor cooperating such that the optical image is spatially correlated to mathematical processing of the data stream with the reduced set filter kernel.

44. The optical imaging system of claim 40, the wavefront coding element comprising an aspheric wavefront coded optical element.

45. The optical imaging system of claim 40, the wavefront coded element comprising constant path profile optics.

46. The optical imaging system of claim 45, the reduced set filter kernel being complimentary to a PSF of the optical image resulting from phase modification of the wavefront through the constant path profile optics.

47. The optical imaging system of claim 40, the image processor implementing a filter tap, for each pixel, with logic consisting of a shifter.

48. The optical imaging system of claim 40, the image processor implementing a filter tap, for each pixel, with logic consisting of a shifter and an adder.

49. The optical imaging system of claim 40, the reduced set filter kernel consisting of power of two coefficients.

50. The optical imaging system of claim 40, the reduced set filter kernel consisting of one or both of sums and differences of power of two coefficients.

51. The optical imaging system of claim 40, the reduced set filter kernel consisting of a plurality or regions wherein at least one of the regions has zero values.

52. The optical imaging system of claim 51, the wavefront coding element modifying the wavefront such that a PSF of the optical image spatially correlates to the regions of the reduced set filter kernel, wherein a substantial amount of information of the PSF is within the final image.

53. The optical imaging system of claim 40, wherein the detector outputs data in the data stream in a non-rectilinear format.

54. The optical imaging system of claim 53, the format comprising one of a diagonal readout and a circular readout.

55. The optical imaging system of claim 53, the reduced set filter kernel being matched to the non-recitilinear format.

56. The optical imaging system of claim 55, the image processor remapping data to a rectilinear format.

57. The optical imaging system of claim 53, information within a PSF of the optical image substantially corresponding to the non-recitilinear format.

58. The optical imaging system of claim 40, the reduced set filter kernel comprising a reduced set gradient filter kernel having a series of single-shift differential taps.

59. The optical imaging system of claim 58, the image processor executing a single summation for any coefficient value of the filter kernel.

60. The optical imaging system of claim 58, the wavefront coding element modifying the wavefront such that a PSF in the optical image contains substantially all information within a spatial pattern corresponding to the filter kernel.

61. The optical imaging system of claim 40, the reduced set filter kernel comprising a reduced set scaling filter kernel and a series of scaling taps.

62. The optical imaging system of claim 61, the image processor executing a single summation for any coefficient value of the filter kernel.

63. The optical imaging system of claim 40, the reduced set filter kernel comprising a reduced set distributive filter kernel and a series of distributive property scale-add taps.

64. The optical imaging system of claim 63, the image processor executing a single summation for any coefficient value of the filter kernel.

65. The optical imaging system of claim 40, the reduced set filter kernel comprising a reduced set distributive filter kernel with a series of distributive property add-scale taps.

66. The optical imaging system of claim 65, the image processor executing a single summation for any coefficient value of the filter kernel.

67. The optical imaging system of claim 40, the reduced set filter kernel comprising a weighted matrix.

68. An image processing system, comprising:

a wavefront coding element that phase modifies a wavefront that forms an optical image;

a color detector for converting the optical image to a data stream; and

an image processor for processing the data stream with a color-specific filter kernel to reverse effects of wavefront coding and generate a final image.

69. The image processing system of claim 68, the image processor comprising means for separating color data from the color digital detector into separate channels.

70. An image processing system, comprising the steps of:

a wavefront coding element to phase modify a wavefront that forms an optical image;

a detector for converting the optical image to a data stream;

means for colorspace converting the datastream;

means for separating spatial information and color information of the colorspace converted datastream into one or more separate channels;

means for deblurring one or both of the spatial information and the color information;

means for recombining the channels to recombine deblurred spatial information with deblurred color information; and

means for colorspace converting the recombined deblurred spatial and color information to generate an output image.

71. The image processing system of claim 70, further comprising means for filtering noise from the data stream.

72. The image processing system of claim 71, the wavefront coding element generating a MTF of the optical image such that spatial correlation exists with the means for filtering noise.

73. The image processing system of claim 70, further comprising means for filtering noise from the recombined deblurred spatial and color information.

74. The image processing system of claim 73, the means for filtering noise from the recombined deblurred spatial and color information utilizing a complement of an MTF of the optical image.

75. The image processing system of claim 70, the means for colorspace converting the datastream comprising a dynamic color space process.

76. The image processing system of claim 75, further comprising means for noise filtering based on the dynamic color space process

77. An electronic device, comprising a camera having:

a wavefront coding element that phase modifies a wavefront that forms an optical image within the camera;

a detector for converting the optical image to a data stream; and

an image processor for processing the data stream with a reduced set filter kernel to reverse effects of wavefront coding and generate a final image.

78. The electronic device of claim 77, wherein the electronic device forms one of a cell phone and teleconferencing apparatus.

79. An image processing method, comprising the steps of:

wavefront coding a wavefront with constant path profile optics that forms an optical image;

converting the optical image to a data stream;

processing the data stream to reverse effects of wavefront coding and generate a final image.

80. An optical imaging system for generating an optical image, comprising:

constant path profile optics that code a wavefront forming the optical image;

a detector for converting the optical image to a data stream; and

an image processor for processing the data stream to reverse effects of wavefront coding and generate a final image.