Reducing ghosting and other image artifacts in a wedge-based imaging system

• US 9,436,980 B2
• Filed: 10/10/2012
• Issued: 09/06/2016
• Est. Priority Date: 10/10/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Powell, Karlton
• Primary Examiner(s): VU, NGOC YEN T
• Application Number:
  US13/648,281
• Publication Number:
  US 20140098245A1
• Time in Patent Office: 1,427 Days
• Field of Search: 348/241
• US Class Current: 1/1
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise
• CPC Class Codes: H04N 25/626 Reduction of noise due to residual charges remaining after image readout, e.g. to remove ghost images or afterimages

Abstract

A computational image processing filter processes an image from a wedge-based imaging system so as to remove artifacts such as blurring and ghost images. By removing the artifacts computationally instead of optically, manufacturing costs and complexity are reduced over prior solutions. In one implementation, the computational image processing filter performs a two-dimensional transform to align a ghost image with a pixel grid defined by the wedge. The transformed image is then stretched using a nonlinear transform to make the ghost pitch versus position a constant. Next, an anti-ghost point spread filter is created and deconvolved. Finally, an inverse of the nonlinear mapping is applied. Artifacts introduced by other optical layers can be reduced by deconvolving the artifact from the image according to a point-spread function representing an effect of the optical layers on the image.

Claims

1. A process comprising:

receiving an image from a sensor into memory of the computer, wherein the sensor is configured to receive output light from a window of an optical wedge and output the image representative of the output light, wherein the optical wedge comprises a face configured to receive light from objects and the window configured to output light representative of the objects;

processing the image with a processor of the computer so as to substantially reduce ghost image data by applying a filter that reverses optical effects of the optical wedge which generate the ghost image data in a wedge-based imaging system by moving ghost pixels from points in the image to points from which the ghost pixels originated; and

outputting the processed image to the memory of the computer.

2. The process of claim 1, wherein outputting comprises outputting the processed image for access by an object processing system to identify one or more objects in the image.

3. The process of claim 1, wherein the wedge based imaging system includes one or more optical layers above the optical wedge which introduces one or more artifacts and wherein processing of the image includes deconvolving the artifact from the image according to a point-spread function representing an effect of the optical layers on the image.

4. The process of claim 1, wherein processing the image comprises:

applying a two-dimensional transform to the image to align the ghost image data with a pixel grid defined by the optical wedge.

5. The process of claim 4, wherein processing the image further comprises:

applying a point-spread filter to the transformed image to substantially remove the ghost image data from the transformed image.

6. The process of claim 5, wherein processing the image further comprises:

applying a transform to the transformed image with the ghost image data substantially removed so as to transform the image back to an original orientation.

7. The process of claim 6, wherein the wedge based imaging system includes one or more optical layers above the optical wedge which introduces one or more artifacts and wherein processing the image includes deconvolving the artifact from the image according to a point-spread function representing an effect of the optical layers on the image.

8. An article of manufacture comprising:

a computer storage medium comprising a memory device or a storage device;

computer program instructions stored on the computer storage medium which, when processed by a processor of a computer, instruct the computer to perform a process comprising;

controlling a wedge-based imaging system connected to the computer to capture an image, the wedge-based imaging system comprising an optical wedge and a sensor, wherein the optical wedge comprises a face configured to receive light from objects and a window configured to output light representative of the objects, wherein the sensor is configured to receive the output light from the window of the wedge and output the image to the computer, the image being representative of the output light from the window of the wedge;

receiving the image from the sensor into memory of the computer;

processing the image with a processor of the computer so as to substantially reduce ghost image data by applying a filter that reverses optical effects of the optical wedge which generate the ghost image data in the wedge-based imaging system by moving ghost pixels from points in the image to points from which the ghost pixels originated; and

outputting the processed image to the memory of the computer.

9. The article of manufacture of claim 8, wherein outputting comprises outputting the processed image for access by an object processing system to identify one or more objects in the image.

10. The article of manufacture of claim 8, wherein the wedge based imaging system includes one or more optical layers above the optical wedge which introduces one or more artifacts and wherein processing the image includes deconvolving the artifact from the image according to a point-spread function representing an effect of the optical layers on the image.

11. The article of manufacture of claim 8, wherein processing the image comprises:

applying a two-dimensional transform to the image to align the ghost image data with a pixel grid defined by the optical wedge.

12. The article of manufacture of claim 11, wherein processing the image further comprises:

applying a point-spread filter to the transformed image to substantially remove the ghost image data from the transformed image.

13. The article of manufacture of claim 12, wherein processing the image further comprises:

applying a transform to the transformed image with the ghost image data substantially removed so as to transform the image back to an original orientation.

14. The article of manufacture of claim 8, wherein computationally reversing the optical effects of the wedge-based imaging system further comprises filtering the image based on presumed locations of ghost data generated by the optical wedge.

15. A wedge-based imaging system, comprising:

an optical wedge having a face configured to receive light from objects and a window configured to output light representative of the objects;

a sensor configured to receive the output light from the window of the wedge and output image data representative of the output light; and

a computational image processing system having an input for receiving the image data from the sensor and an output providing the image data with artifacts substantially reduced by applying a filter that reverses optical effects of the optical wedge which generate the ghost image data in the wedge-based imaging system by moving ghost pixels from points in the image to points from which the ghost pixels originated.

16. The wedge-based imaging system of claim 15, wherein outputting comprises outputting the image for access by an object processing system to identify one or more objects in the image.

17. The wedge-based imaging system of claim 15, wherein outputting comprises outputting the image for display on a display.

18. The wedge-based imaging system of claim 15, wherein processing the image comprises:

applying a two-dimensional transform to the image data to align ghost image data by position.

19. The wedge-based imaging system of claim 18, wherein processing the image further comprises:

applying a point-spread filter to the transformed image to substantially remove ghost data from the transformed image.

20. The wedge-based imaging system of claim 19, wherein processing the image further comprises applying a transform to the transformed image with the ghost data substantially removed so as to transform the image back to an original orientation.

21. The wedge-based imaging system of claim 19, wherein processing the image further comprises applying a transform to the transformed image with the ghost data substantially removed so as to transform the image back to alignment with the original input image.

22. The wedge-based imaging system of claim 21, further comprising one or more optical layers above the optical wedge and that introduces one or more artifacts, and wherein the processing includes deconvolving the one or more artifacts from the image according to a point-spread function representing an effect of the optical layers on the image.

23. The wedge-based imaging system of claim 19, wherein processing the image further comprises applying a transform to the transformed image with the ghost data substantially removed so as to transform the image back in its original grid.

24. The wedge-based imaging system of claim 15, wherein processing the image further comprises filtering the image based on presumed locations of ghost data.

25. The wedge-based imaging system of claim 15, further comprising one or more optical layers above the optical wedge and that introduces one or more artifacts, and wherein the processing includes deconvolving the one or more artifacts from the image according to a point-spread function representing an effect of the optical layers on the image.