METHOD AND APPARATUS FOR MOTION INVARIANT IMAGING

• US 20090244300A1
• Filed: 03/28/2008
• Published: 10/01/2009
• Est. Priority Date: 03/28/2008
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Levin, Anat, Sand, Peter, Cho, Taeg Sang, Freeman, Willliam T., Durand, Fredo
• Granted Patent: US 8,451,338 B2
• Time in Patent Office: Days
• US Class Current: 348/208.500
• CPC Class Codes: G03B 17/00 Details of cameras or camera bodies; Accessories therefor lens hoods or caps G03B11/04
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction

Abstract

Object motion during camera exposure often leads to noticeable blurring artifacts. Proper elimination of this blur is challenging because the blur kernel is unknown, varies over the image as a function of object velocity, and destroys high frequencies. In the case of motions along a 1D direction (e.g. horizontal), applicants show that these challenges can be addressed using a camera that moves during the exposure. Through the analysis of motion blur as space-time integration, applicants show that a parabolic integration (corresponding to constant sensor acceleration) leads to motion blur that is not only invariant to object velocity, but preserves image frequency content nearly optimally. That is, static objects are degraded relative to their image from a static camera, but all moving objects within a given range of motions reconstruct well. A single deconvolution kernel can be used to remove blur and create sharp images of scenes with objects moving at different speeds, without requiring any segmentation and without knowledge of the object speeds.

Claims

1. A method for deblurring images, comprising:

during imaging of a moving object, blurring an entire scene that includes the moving object, said blurring being invariant to velocity of the moving object;

deconvoluting the blurred entire scene and generating a reconstructed image, the reconstructed image bearing the moving object in a deblurred state.

2. A method of claim 1 wherein the step of blurring the entire scene is implemented by moving any one or combination of a camera, a lens element of the camera and sensor of the camera.

3. A method of claim 2 wherein the step of moving includes movement in a range of direction and speed sufficient to include direction and speed of the moving object.

4. A method as claimed in claim 3 wherein movement is any of:

a sinusoidal pattern,

a parabolic pattern,

a simple harmonic motion, and

a sweep motion starting in one direction and progressively decelerating then accelerating in an opposite direction.

5. The method of claim 4 wherein the movement follows a parabolic path by moving laterally initially at a maximum speed of the range and slowing to a stop and then moving in an opposite direction laterally, increasing in speed to a maximum speed of the range in the opposite direction and stopping.

6. A method as claimed in claim 1 wherein the moving object is a moving vehicle on a roadway.

7. A method as claimed in claim 1 wherein the moving object is a body part of a patient (human or animal).

8. A method as claimed in claim 1 wherein the deconvolution is computed near real-time of image exposure (i.e. during imaging of the moving object).

9. A method as claimed in claim 1 wherein the deconvolution is by an application separate from and subsequent to image exposure by a camera.

10. A method as claimed in claim 1 wherein the step of blurring blurs the entire scene, including static objects and the moving object, using a single point spread function which is invariant to object motion such that the deconvoluting removes blur from the blurred entire scene using the point spread function, without segmenting the moving object and without estimating velocity of the moving object.

11. A method as claimed in claim 1 wherein the reconstructed image bears multiple moving objects, each in a deblurred state.

12. Apparatus for deblurring images, comprising:

a camera controller, during imaging of a moving object the controller effectively moving a camera to blur an entire scene that includes the moving object and said blurring being in a manner that is invariant to velocity of the moving object; and

a deconvolution processor coupled to receive the blurred entire scene and deconvoluting the blurred entire scene, resulting in a reconstructed image bearing the moving object in a deblurred state.

13. Apparatus as claimed in claim 12 where the controller effectively moving the camera moves any one or combination of the camera, a lens element of the camera and a sensor of the camera.

14. Apparatus as claimed in claim 12 wherein the controller effectively moving the camera includes movement in a range of direction and speed sufficient to include direction and speed of the moving object.

15. Apparatus as claimed in claim 14 wherein movement is any of:

a sinusoidal pattern,

a parabolic pattern,

a simple harmonic motion, and

a sweep motion starting in one direction and progressively decelerating then accelerating in an opposite direction.

16. Apparatus as claimed in claim 15 wherein the movement follows a parabolic path by moving laterally initially at a maximum speed of the range and slowing to a stop and then moving in an opposite direction laterally, increasing in speed to a maximum speed of the range in the opposite direction and stopping.

17. Apparatus as claimed in claim 12 wherein the moving object is any of a moving vehicle on a roadway, a body part of a patient, and a face of a subject.

18. Apparatus as claimed in claim 12 wherein the deconvolution processor deconvolutes the blurred entire scene near real-time of image exposure.

19. Apparatus as claimed in claim 12 wherein the deconvolution processor executes as an application separate from and subsequent to image exposure by the camera.

20. Apparatus as claimed in claim 12 wherein the reconstructed image bears multiple moving objects, each in a deblurred state.

21. A motion invariant imaging system comprising:

controller means for effectively controlling a camera during imaging of a moving object, and blurring an entire scene that includes the moving object; and

deconvolution means for deconvoluting the blurred entire scene and generating a reconstructed image bearing the moving object in a deblurred state;

wherein the controller means through the camera blurring the entire scene includes blurrying static objects and the moving object using a single point spread function which is invariant to object motion, such that the deconvoluting removes blur from the blurred entire scene using the point spread function, without segmenting the moving object and without estimating velocity of the moving object.